In practice, feature engineering is both science and a bit of witchcraft. It often involves both iteration and experimentation to uncover hidden patterns and relationships within the data. For instance, a data scientist might transform raw sales data into features such as average purchase value, purchase frequency, or customer lifetime value, which can significantly boost the performance of a churn prediction model. By thoughtfully engineering features, practitioners can provide machine learning models with the most informative inputs, ultimately leading to better accuracy and more robust predictions.
What’s more?
Incorporate more and more data sources
Feature engineering platform
What is data engineering
As we mentioned above, feature engineering is certainly a subset of data engineering. It involves the ingestion of data from a source, applying a series of transformations, and making the final result available to be queried by a model for training purposes. You can construct feature engineering pipelines to resemble data engineering pipelines, having schedules, specific source and sink destinations, and availability for querying. However, this configuration would only really apply once you have surpassed the experimentation stage and determined a need for a consistent flow of new feature data.
What is feature engineering
Image description
1. Functions
Functionally, there is nothing to differentiate data vs features - data points (link). Where feature engineering and data engineering really differ is in the objectives and motivations for constructing the pipelines. In general, data engineering serves a broader, more unified purpose than feature engineering. Data engineering platforms are constructed to be flexible and universal, ingesting various types and sources of data into a unified storage location where any number of transformations and use cases can be applied. The intent of a well constructed fact table or gold layer in a data lake is to provide a single source of truth that answers many different questions, produces many reports, and can be consumed by many downstream customers.
2. Practise
And in practice, an organization’s data engineering team will be responsible for the curation and maintenance of all data pipelines, not just those that relate to machine learning. These pipelines may power BI dashboards used by C-Suite, auditing reports that feed payroll, or event logs that show a user’s history of actions within the application. Feature engineering, on the other hand, serves a specific purpose, finding the tailored inputs and columns that will generate the best predictive results for a machine learning model. Data scientists and machine learning engineers are not tasked with developing a universal data model that will ingest all data points throughout an organization, they just need to select, curate, and clean the data needed to power their models.
3. Machine learning
Now, as machine learning teams grow and begin to incorporate more and more data sources into their models, their feature engineering platform may start to resemble a larger data engineering platform in the tools and methodologies they employ. But, the intent is not to establish flexible data models that can be used throughout the organization - it is simply to power their machine learning models.
Understanding interactive social engineering
Social engineering describes a broad set of attack mechanisms that target either specific individuals or roles within an organization. Interactive social engineering attacks are a subset characterized by direct engagement between an attacker and their target across voice, chat, service tickets, email threads, or other communication channels. The attacker uses the interaction to coerce the target into disclosing information or taking another sensitive action. Traditionally, the parties are human, but agentic AI systems have started to replace and supplement human attackers and defenders.
These attacks are common and deceptively simple. Most follow the same basic structure, in which an attacker contacts a target, often under a false identity, and makes a request that sounds routine, such as a password reset. Narrative details can change as the conversation progresses. A successful attempt can lead to outcomes such as account takeover, data theft, or extortion.
Across incidents, attackers can change names, phone numbers, and other details while relying on the same underlying mechanisms. Recognizing recurring patterns requires a consistent way to break each interactive social engineering attack into its component parts. Without that structure, differences in incident reporting can obscure recurring attack mechanisms and opportunities for defense.
About this document
Humanix developed this analysis of interactive social engineering attacks to provide a structure to analyze and describe social engineering techniques and procedures. It describes attacks across six dimensions, from initial contact through post-compromise activity. Table 1, below, defines the dimensions; the recorded values form an attack pattern.
This structure complements MITRE ATT&CK’s coverage of social engineering under Technique T1684. ATT&CK classifies the adversary technique, while Humanix’s approach provides a consistent way to describe its use in particular attacks.
Examining an attack pattern alongside organizational records and approved procedures can help analysts establish how an attack unfolded, whether verification steps were followed, and where defenses could have intervened. Patterns can also be compared across incidents to identify recurring mechanisms, even when phone numbers, domains, or other indicators differ. Those findings can inform prevention, detection, and response priorities.
Table 1: Interactive social engineering attack patterns by dimension
Many of these dimensions produce observable indicators in voice transcripts, emails, SMS messages, and other communication records. Others can be established by comparing those communications with identity, access, ticketing, endpoint, network, and other organizational records. Identifying these indicators within available records allows defenders to assess whether an interaction is consistent with a known social engineering attack pattern and detect related attacks even when individual indicators change.
Using this document
This document provides a consistent way to examine an attack or attempted attack. Analysts should use available evidence from incident reports and organizational records to characterize each of the six dimensions. Some details may remain unknown when the evidence is incomplete.
Examining the relationships among these dimensions helps explain how the attack unfolded and what the attacker sought to accomplish. Security teams can then assess the interaction against applicable procedures and controls to determine which safeguards were in place, whether required procedures were followed, and where the attack could have been interrupted or detected.
The resulting patterns can inform the selection of preventive measures and the development of detection and response capabilities.