Video analysis is the process of automatically extracting information and insight from videos. Typical examples include detecting, recognizing and tracking objects, often with deep understanding of objects such as faces and text or comprehensive understanding of the video by converting speech to text or classifying scenes and actions.  Together with a complete suite of high-level image and audio analysis functions and tightly integrated with machine learning and neural networks, Wolfram Language enables customizable workflows for analyzing videos from different areas.

Video Visualization

VideoSummaryPlot — plot a summary of a video

HighlightVideo — highlight detected features or tracked objects

AudioPlot  ▪  Spectrogram  ▪  VideoFrameList

Object Detection & Recognition

FindFaces — find faces in video frames

ImageBoundingBoxes — find objects in video frames

ImagePosition  ▪  ImageCases  ▪  ImageContents  ▪  ImageContainsQ  ▪  TextRecognize  ▪  FindImageText  ▪  BarcodeRecognize  ▪  FacialFeatures

Object & Feature Tracking

VideoObjectTracking — track objects in a video

VideoStabilize — stabilize shaky videos

ObjectTrackingData  ▪  ImageDisplacements  ▪  ImageFeatureTrack

Machine Learning & Neural Networks »

"VideoFrames" — video frames net encoder

FeatureExtract  ▪  FeatureSpacePlot  ▪  NetModel  ▪  NetTrain  ▪  ...

Models from the Wolfram Neural Net Repository »

"FastDVDNet Trained on DAVIS Data" (video denoising)  ▪  "3D-Inflated ResNet-50 Trained on Kinetics 400 Data" (action recognition)  ▪  "ColorNet Image Colorization Trained on ImageNet Competition Data" (colorize frames)  ▪  …

Frame-by-Frame Analysis

VideoIntervals — find video intervals of interest

VideoMapTimeSeries  ▪  VideoMapList

Frame-by-Frame Segmentation

ImageSegmentationComponents — find high-level segments in video frames

MorphologicalComponents — find connected components in video frames

ClusteringComponents  ▪  WatershedComponents  ▪  GrowCutComponents  ▪  ImageForestingComponents

Frame-by-Frame Feature Detection

ImageCorners — find corner points in video frames

ImageKeypoints  ▪  ImageLines  ▪  FindImageShapes

Audio Visualization

AudioPlot — waveform plot of audio

Spectrogram  ▪  Periodogram  ▪  Cepstrogram

Audio Analysis »

AudioIntervals — return intervals of silence, audio, etc.

AudioMeasurements  ▪  AudioLocalMeasurements  ▪  ShortTimeFourier  ▪  ...