High-dimensional neural network — SIMPLE-NN 20.2 documentation

Function related parameter¶

train: (boolean, default: true) If true, training process proceeds.
test: (boolean, default: false) If true, predicted energy and forces for test set are calculated.
continue: (boolean or string, default: false) If true, training process restarts from save file (SAVER.*, checkpoints). If continue: weights, training process restarts from the LAMMPS potential file (potential_saved).

Note

potential_saved only contains the weights and bias of the network. Thus, hyperparameter used in Adam optimizer is reset with continue: weights.

Network related parameter¶

nodes: (str or dictionary, default: 30-30) String value to indicate the network architecture. 30-30 means 2 hidden layers and each hidden layer has 30 hidden nodes. You can use different network structure for different atom types. For example:
```
nodes:
  Si: 30-30
  O: 15-15
```
regularization: (dictionary) Regularization setting. Currently, L1 and L2 regularization is available.
```
regularization:
  type: l2 # l1 and l2 is available
  params:
    coeff: 1e-6
```
use_force: (boolean, default: false) If true, both energy and force are used for training.
force_coeff: (float, default: 0.1) Ratio of force loss to energy loss in total loss
use_stress: (boolean, default: false) If true, both energy and stress are used for training. (Caution : The unit of stress RMSE written in LOG file is kbar.)
stress_coeff: (float, default: 0.00001) Ratio of stress loss to energy loss in total loss
stddev: (float, default: 0.3) Standard deviation for weights initialization.

PCA-related parameters¶

pca: When set to true, PCA is applied to the input descriptor vector.

Default:
```
pca: false
```
pca_whiten: When set to true, PCA-transformed vector is whitened (the variances of principal components are normalized to unity). The whitening is different than original scheme. See below for more details on the difference.

Default:
```
pca_whiten: true
```

pca_min_whiten_level: This option can be used to suppress whitening of principal components with small variances. Originally, whitening normalize variances of all principal components to unity:

\[PC_{i,\text{whiten}}=\frac{PC_i}{\sqrt{\text{Var}_i}}\]

A small constant \(a\) is added to the variance to suppress principal components with small variances:

\[PC_{i,\text{whiten}}=\frac{PC_i}{\sqrt{\text{Var}_i + a}}\]

In practice, this can be used to reduce the generalization error.

Default:
```
pca_min_whiten_level: 1.0e-8
```

Optimization related parameter¶

method: (str, default: Adam) Optimization method. You can choose Adam or L-BFGS.
batch_size: (int, default: 64) The number of samples in the batch training set.
full_batch: (boolean, default: true) If true, full batch mode is enabled.

Note

In the full_batch mode, batch_size behaves differently. In full_batch mode, the entire dataset must be considered in one iteration, but this often causes out of memory problems. Therefore, in SIMPLE-NN, a batch dataset with the size of batch_size is processed at once, and this process is repeated to perform operations on the entire data set.

total_iteration: (int, default: 10000) The number of total training iteration. If negative, early termination scheme is activated (See break_max below).
learning_rate: (float, default: 0.0001, Exponential decay) Learning rate for gradient descendent based optimization algorithm.
force_coeff and energy_coeff: (float, default: 0.1 and 1., Exponential decay) Scaling coefficient for force and energy loss.
loss_scale: (float, default: 1.) Scaling coefficient for the entire loss function.
optimizer: (dictionary) additional parameters for user-defined optimizer

Logging & saving related parameters¶

show_interval: (int, default: 100) Interval for printing RMSE in LOG file.
save_interval: (int, default: 1000) Interval for saving the neural network potential file.
save_criteria: (list, default: []) Criteria for saving the neural network potential file. Energy error for validation set (v_E), force error for validation set (v_F), and force error for validation set for sparsely sampled training points (v_F_XX_sparse) are possible. A network is saved only when all values in the criteria are smaller than previous save points. If not, break_count is increased (See break_max below).

Note

In SIMPLE-NN, save conditions(save_interval and save_critera) are checked every multiple of show_interval. Thus, it is recommended to set save_interval to multiples of show_interval.

Note

Every multiple of show_interval, SIMPLE-NN calculates energies and forces for entire validation set. so the process takes a lot of time in general. Thus, small show_interval may slow down the training speed.

break_max: (int, default: 10) If save criteria is not satisfied in current save points, break_count increases. Optimization process is terminated when break_count >= break_max. This tag is only activated when total_iteration is negative.
print_structure_rmse: (boolean, default: false) If true, RMSEs for each structure type are also printed in LOG file.

Performance related parameters¶

inter_op_parallelism_threads and intra_op_parallelism_threads: (int, default: 0, 0) The number of threads for CPU. Default is 0, which results the values set to the number of logical cores. The recommended values are the number of physical cores for intra_op_parallelism_threads and the number of sockets for inter_op_parallelism_threads. intra_op_parallelism_threads should be equal to OMP_NUM_THREADS.
cache: (boolean, default: false) If true, batch dataset is temporarily saved using caches. Calculation speed may increase but larger memory is needed.

Exponential decay¶

Some parameters in neural_network may need to decrease exponentially during the optimization process. In those cases, you can use this format instead of float value. More information can be found in Tensorflow homepage

parameter_name:
    learning_rate: 1.
    decay_rate: 0.95
    decay_steps: 10000
    staircase: false

Note

If continue: true, global_step (see the link above) of save points is also loaded. Thus, you need to consider the global_step to calculate the values from exponential_decay. On the contrary, global_step is reset when continue: weights

High-dimensional neural network¶

Introduction¶

Parameters¶

Exponential decay¶

Methods¶