Watch: From Raw Video to Deep Learning in 68 seconds

How do you go from raw video, to training machine learning models

Traditionally, video data in machine learning is treated as separate images, sampled from the video (sometimes at very low frame rates, one or two images per second), and then processed as images. This can seem counter-intuitive, since you’d want as much training data as possible. For example, a minute of video can have 1800 meaningful images with a wealth of data, why would you use only 60 images and discard of 96.6% of your potential training data? Why would you ignore the temporal data in the video?

Read below to learn about a straightforward treatment of video to get training data, using all the frames in the video. The training data is immediately available and can be directly used in your machine learning code, eg through a Jupyter notebook. You can also watch a webinar we recently hosted demonstrating the same approach.

What should I do with my raw video?

Using our platform, creating training data from raw video becomes simpler; there’s no need to extract images from the video and thus lose most of your data; automatic tracking for objects makes it a breeze to to annotate hundreds of frames accurately; you can annotate videos yourself for a quick test, or have dozens of workers continuously go over large amounts of video files. Lastly, the training data is available immediately in multiple formats and ready to train in your ML algorithms.

Watch the webinar

Recently Clay Sciences hosted a webinar which showed all three steps. During the webinar, we live-demoed our annotation platform and annotated a video of cars driving in New York’s Times Square; then we exported the annotation data as a json file, and used it in a Jupyter notebook to load a pre-trained RESNET34 CNN model (using the fastai framework), transfer-learned the new annotations onto this network, and saved the new model.

To watch the webinar, fill in your details below:

About Clay Sciences
Clay Sciences accelerates the process of building machine learning models, providing a platform for data scientists for obtaining training data quickly, efficiently and at scale.

Ready to get started? Send us an email at

Subscribe to our mailing list

* indicates required