bsmart.scans.MLScanner.MLS_DNNC

MLScanner DNNC method

MLScanner method MLS_DNNC based on code from

This scan implements an active learning strategy using a Deep Neural Network Classifier (DNNC) to efficiently find “good” points in a parameter space. A point is considered “good” if the likelihood is below a specified threshold. This is a generalisation of the original algorithms; in the original package the scan looked for a primary observable to compare to a threshold. Since the likelihood can be set as “EXPUSER” for a given observable, the original case can also be accommodated – but a likelihood is more generally useful.

The process is as follows:

  1. Initialization: The scan begins by evaluating a small set of randomly generated points (Bootstrap_Points). It can also load an initial dataset from a CSV file (InitCSV).

  2. Initial Training: A DNN classifier is trained on this initial dataset. The points are labeled as “good” (1) or “bad” (0) based on the Threshold_Value. The loss function can be weighted to handle the initial class imbalance.

  3. Active Learning Loop: The scan enters a loop to iteratively discover new good points until a Target_Points count is reached. In each iteration:

    1. A large number of Candidate_Points are randomly generated.

    2. The trained DNN predicts which of these candidates are likely to be good.

    3. The most promising candidates, plus a small Random_Fraction, are selected for evaluation by the physics code.

    4. SMOTE for Imbalance: If too few good points are found, the Synthetic Minority Over-sampling TEchnique (SMOTE) is used to generate synthetic good points to balance the dataset and improve classifier performance in sparse regions.

    5. Retraining: The DNN is retrained with the newly discovered points, becoming progressively better at identifying promising regions of the parameter space.

  4. Data Collection: All discovered good points and their observable values are saved to a CSV file.

This method is particularly effective for high-dimensional parameter spaces where exhaustive scanning is computationally prohibitive.

Information

BSMArt Name: MLS_DNNC

Requires:

  • imblearn

  • sklearn

  • torch

  • pandas

Settings:

Networks

  • Iterations: Number of active learning iterations (default: 10).

  • Candidate_Points: Number of candidate points to generate and score in each iteration (default: 500).

  • Bootstrap_Points: Number of initial random points to evaluate (default: 100).

  • Points_Per_Iteration: Number of candidate points to evaluate in each iteration (default: 300).

  • Kinitial: Number of neighbors for the SMOTE algorithm (default: 5).

  • Threshold_Value: The threshold for the primary observable to classify a point as ‘good’ (default: 100).

  • Random_Fraction: Fraction of points per iteration to be selected randomly, for exploration (default: 0.2).

  • Batch Size: Batch size for training the neural network (default: 500).

  • Neurons: Number of neurons in the hidden layers of the DNN (default: 100).

  • Epochs: Number of epochs for training the DNN in each cycle (default: 250).

  • LearningRate: Learning rate for the AdamW optimizer (default: 1e-2).

  • Verbose: Verbosity level (default: 0).

Setup

  • InitCSV: Path to an optional CSV file with initial points to seed the scan.

  • Points: Number of points to generate in total before stopping (default: 1000)

class bsmart.scans.MLScanner.MLS_DNNC.NewScan(inputs, log)[source]

Bases: Scan

extract_from_valid_points(valid_points)[source]
get_losses(observables)[source]

Returns a list of losses.

The C-function loss should be zero if the observable is within allowed bounds, and greater than zero outside it.
If we include a validity condition, then other observables, evaluated afterwards, will be returned as NaN.

-> We should assign these the maximum loss ~ 710.

This is then compatible with the explicit use of ‘hierarchical observables’, where all other observables are set to the maximum.

initialise()[source]

Need to make sure we override certain settings

likelihood_fns

self.header = [] if ‘Observables’ in self.inputs:

for observable_name in self.inputs[‘Observables’]:

self.header.append(observable_name)

if len(self.header) == 0:

sys.exit(“No observables found in inputs!”)

self.primary_observable = self.header[0]

loss_fn_selector(training_labels)[source]
network_architecture_selector(n_training_points)[source]
observable_masks

self.header = [] if ‘Observables’ in self.inputs:

for observable_name in self.inputs[‘Observables’]:

self.header.append(observable_name)

if len(self.header) == 0:

sys.exit(“No observables found in inputs!”)

self.primary_observable = self.header[0]

postprocess(Point, observables, data_point, temp_dir, log, lock=None)[source]

return the likelihood; we won’t get this far if the point failed to be generated

run()[source]
smooth_cap_loss(x)[source]

Caps the loss by applying a sigmoid. This is useful for losses that are unbounded.

bsmart.scans.MLScanner.MLS_DNNC.generate_param_points(inputs, num_points)[source]
bsmart.scans.MLScanner.MLS_DNNC.labeler(obs_values, threshold)[source]
bsmart.scans.MLScanner.MLS_DNNC.torchfit(X, y, model, criterion, optimizer, epochs, batch_size, verbose=True, patience=10, min_delta=1e-05)[source]