Some video clips, regardless of how they were taken, may contain only scenes of the same background and objects of related actions. However, most video clips will likely contain scenes of varying similarity and vastly different backgrounds and actions taking place. Segments of similar scenes will most probably contain similar information, such that a single frame (picture) from that particular video segment can represent most of the information in the entire segment.

Thus, it is more effective to divide a video clip into separate segments containing similar background and actions when it comes to performing digital analysis on the video clip. Each such segment is referred to as a video shot and the most representative frame (picture) that can be extracted and analyzed on is referred to as a keyframe.

Therefore, the objective of VSS is to analyze the video file, determine the video segment(s) containing continuous action in space and time and generate the relevant keyframes.