Optimized work with growing files

Whitepaper

Live broadcasts, such as sporting events, live concerts, or television programs featuring a repeat broadcast immediately following the initial broadcast, have unique editing requirements. For these events, broadcasters don’t want to wait until the entire recording is captured before editing begins. For example, a sports broadcast or live event may still be in process, but the post-production team needs to create highlights of pivotal plays or key moments as quickly as possible.

In these cases, editors have the option of working with growing files, which are used for recordings where editing needs to begin before a recording ends. Adobe Premiere Pro enables editors to grab footage and edit highlights or make other adjustments to video that has already been captured while a recording is still in progress.

Which parameters determine whether Adobe Premiere Pro classifies a file as growing?

When importing a file, Premiere Pro checks whether it already has a footer, which is usually appended to the data block at the end of the file-writing process. In addition, various file formats, such as MXF, also have an internal flag that specifies a value for the current status of the file, such as growing, open, closed, or complete. The file size is measured and compared at regular intervals, and if the size changes, the file is still growing. If this is the case, Premiere Pro classifies the file as growing.

Which file formats does Adobe (Premiere Pro) support?

Officially, Adobe supports the following formats:

• AVC-Intra Class 50/100 – MXF OP-1a
• IMX 30/40/50 – MXF OP-1a
• XDCAM HD 50/35/25/18 RDD9 (low latency version of XDCAM HD is excluded)
• OP-1B formats

It is also possible to work with the following formats:

• AVC-Intra Class 50/100 – QuickTime MOV
• IMX 30/40/50 – QuickTime MOV
• XDCAM HD MOV (In the case of QuickTime MOV, the writing of XMP data must be deactivated in Premiere Pro, as Premiere tries to write the metadata into the growing QuickTime file. In this case, however, this is already blocked by the blocked by the writing instance.)
• Video .ts steams
• MPEG# via 3rd party plugins

Growing files are also supported when proxies are attached to hi-res media.

What are the challenges when working with growing files?

Automatic updates during the editing process Premiere Pro updates growing files at an automatic interval that can be set by the user. For example, if the file is updated every 10 seconds, the new part recorded during those 10 seconds is added. The addition of the newly recorded content causes a short update pause in the user interface as the file refreshes.

If you are working with several growing files in a project, there are even more frequent refresh pauses, because the refresh always starts exactly at the interval corresponding to the timestamp when the file was imported. As a result, the individual clips have different refresh times, which can disrupt an editor’s workflow.

A further complication occurs when a growing clip undergoes a header change in the creation process, such as when a header change is made after the initial import. This can lead to Premiere Pro misinterpreting the growing clip, resulting in "Danger Stripes", because the IndexTable is no longer correct. When this occurs, it may be necessary to delete the cache files and reconnect the growing clip to Premiere Pro.

How can you optimize working with growing files?

Users should avoid long GOP formats when dealing with growing clips, as every refresh requires the file to be scanned and partially decoded to create a new index table. Using Intraframe codecs is preferable.

Another significant influence is the file system where the growing clip is located. The faster the file system, the faster the indexing can take place, resulting in better read performance of the client and faster data throughput. Optimized MXF read block sizes can also help to optimize fast access to the data.

What storage is best for working with growing files?

• StorNext (with limitations for resharing via SMB)
• Editshare EFS
• Mediagrid
• Local file systems
• SMB, V1 or V3
• Growing files in Adobe Premiere Pro

What solution does MoovIT offer to optimize working with growing files?

To make working with growing clips in Adobe Premiere Pro more convenient, Adobe SI partner MoovIT has developed the Refresh Control plug-in. This was used for the first time in collaboration with Adobe at the 2013 Confederations Cup in Belo Horizonte. As with most sporting events, editing began while the footage was still being recorded.

Refresh Control allows a manual refresh process instead of auto-refresh for growing clips. This has the distinct advantage for the editor of allowing a frictionless work process, as the editor can use the plug-in to determine when to refresh. To do this, the auto-refresh process in Premiere Pro must be deactivated. This can be changed manually in the Premiere Pro settings or automated if you are working with Helmut4. Instead of auto-refresh, the growing file update is now initiated by clicking on the "Refresh Hedgehog".

This not only allows the editor to decide when to refresh, but also which file to refresh when working with multiple growing clips. Not all growing clips in the project are refreshed, only the clip loaded into the source player, so content is only retrieved at the point where it is needed.

Technically, this implementation was made possible by the free API interface in Premiere Pro. Adobe makes all APIs available in the form of the ExtendScript, so MoovIT was able to develop a plug-in with Refresh Control that directly addresses the refresh function for growing clips without affecting other Premiere Pro functions.