1. Home
  2. Classes
  3. TensorflowVGGish

new TensorflowVGGish()

Description

Class with methods for computing inference of Essentia-Tensorflow.js VGGish-based pre-trained models. The predict method expect an input audio feature computed using EssentiaTFInputExtractor.

Examples
// FEATURE EXTRACTION
			// Create `EssentiaTFInputExtractor` instance by passing
			// essentia-wasm import `EssentiaWASM` global object and `extractorType=vggish`.
			const inputFeatureExtractor = new EssentiaTFInputExtractor(EssentiaWASM, "vggish");
			// Compute feature for a given audio signal array
			let inputVGGish = inputFeatureExtractor.computeFrameWise(audioSignal);
			// INFERENCE
			const modelURL = "./classifiers/danceability/danceability-vggish-audioset-1/model.json"
			// Where `tf` is the global import object from the `@tensorflow/tfjs*` package.
			const vggish = new TensorflowVGGish(tf, modelURL);
			// Promise for loading the model
			await vggish.initialize();
			// Compute predictions for a given input feature.
			let predictions = await vggish.predict(inputVGGish);
Details

EssentiaTensorflowJSModel

machinelearning/tfjs_models.js, line 304

Methods

<async> initialize()

Description

Promise for loading & initialise an Essentia.js-TensorFlow.js model.

Details

EssentiaTensorflowJSModel#initialize

machinelearning/tfjs_models.js, line 98

arrayToTensorAsBatches( inputFeatureArray, inputShape, patchSize [, zeroPadding ] ) → {tf.Tensor3D}

Description

Converts an input 1D or 2D array into a 3D tensor (tfjs) given it's shape and required patchSize. If padding=true, this method will zero-pad the input feature.

Parameters
Name Type Attributes Default Description
inputFeatureArray Float32Array | Array.<any>

input feature array as either 1D or 2D array
inputShape Array.<any>

shape of the input feature array in 2D.
patchSize number

required patchSize to dynamically make batches of feature
zeroPadding boolean <optional>
 false

whether to enable zero-padding if less frames found for a batch.
Returns

returns the computed frame-wise feature for the given audio signal.

Details

EssentiaTensorflowJSModel#arrayToTensorAsBatches

machinelearning/tfjs_models.js, line 126

predict( inputFeature [, zeroPadding ] ) → {array}

Description

Run inference on the given audio feature input and returns the activations

Parameters
Name Type Attributes Default Description
inputFeature InputVGGish

audio feature required by the VGGish model.
zeroPadding boolean <optional>
 false

whether to do zero-padding to the input feature.
Returns

activations of the output layer of the model

Details

machinelearning/tfjs_models.js, line 317