Model Building

How Model Building Works

Model building turns raw data into a machine learning model by choosing what information to learn from, selecting an algorithm, and iterating on training so the model can generalise beyond the examples it has seen. The process is usually experimental, because small choices in data quality, labeling, and hyperparameters can materially change accuracy, bias, and robustness.

It is not just a data science workflow, it is also a control point for governance. The definition already on the page notes that model building includes checking data provenance, training access, and whether the resulting model is approved for use, which makes the activity part of the broader security and approval chain around AI systems.

What Changes During Training and Validation

Model building typically moves through feature selection, algorithm choice, parameter tuning, and evaluation. Feature selection shapes what the model is allowed to learn, while parameter tuning changes how the model fits the data. Validation then tests whether the model performs acceptably on data it did not train on, which helps reveal overfitting and weak generalisation before deployment.

The important distinction is that training performance alone is not enough. A model can score well on its training set and still behave poorly in production if the data is unrepresentative, the labels are noisy, or the evaluation set is too similar to the training set. Good model building therefore depends on clear separation between training, validation, and test data, plus enough traceability to explain what changed between iterations.

Why Provenance and Approval Matter

Because model building depends on data, the trustworthiness of the final model is constrained by the trustworthiness of the inputs. If training data is incomplete, manipulated, stale, or drawn from an undocumented source, the model may inherit errors that are hard to detect after deployment. The same is true when the build process itself lacks control over who can train, modify, or approve a model.

That is why governance is part of the subject, not an afterthought. Organisations need to know which datasets were used, how they were prepared, who had access during training, and whether the resulting artifact is authorised for the intended use. For teams building application security or AI governance programmes, the Application Security Guide is a useful companion for understanding how build-time assurance, testing, and approval fit into a broader secure development lifecycle.

How Model Building Affects Downstream Risk

A model is only as dependable as the process that created it. Poorly controlled model building can lead to brittle performance, hidden bias, unsafe automation, or the use of an unapproved model in a production workflow. It can also create a supply-chain style dependency on data, code, notebooks, and training infrastructure that may be reused across multiple systems.

In practice, the biggest downstream concern is that a model built with weak controls may look legitimate while still carrying hidden defects. That makes post-build review important, especially when a model is reused, fine-tuned, or promoted into a higher-impact environment without fresh evaluation.

Risk and Threat Considerations

Model building has a meaningful risk dimension because weaknesses in the training process can produce models that are inaccurate, unsafe, or untrustworthy even before deployment. If attackers, insiders, or careless operators can influence training data, parameter choices, or approval steps, they can shape the model’s behaviour in ways that are difficult to notice later.

Failure mechanism: Bad data, poisoned data, uncontrolled training access, or weak validation can create a model that appears sound but embeds flawed or adversary-influenced behaviour.

Impact: The organisation may deploy a model that makes poor decisions, leaks sensitive patterns, amplifies bias, or fails in ways that are expensive to detect and correct after use.

Related resources from NHI Mgmt Group

• What should organisations do before building a graph-based identity model?
• How should SMEs start implementing PAM without building an enterprise SOC model?
• How do security teams compare model cost, latency, and output quality across providers without building a separate evaluation workflow?
• Which global privacy regulations should enterprise teams plan for when building a compliance operating model?