Life after Digidelivery - Gobbler?

I have been testing a new service recently that looks like a reasonable replacement for Digidelivery now that Aspera have announced the death-knell. It is called Gobbler and allows you to back up your Pro Tools sessions to an Amazon S3 storage system (cloud based/remote, replicated backup).

It’s incredibly simple to setup and use and allows you to then share backed-up projects with others. It can also keep a offline record of where your sessions reside, scan your home-based drives and it will catalog all of the audio files associated with the project, the geographic location of those files and give you an idea of the size of the session. 

I plan to continue testing this system as it moves out of it’s initial beta period next Monday (June 27th 2011). During this time however if you sign up you get 50GB of free storage to start backing up and sharing your sessions. 

To take advantage of this now click on this link to learn more.

FULL DISCLOSURE: Clicking this link helps me to get more free space for referrals but I do genuinely believe this could be a viable replacement for Digidelivery in small post houses/independent editors.

 

Project File Type support copied from the https://www.gobbler.com/features#organize site:

Compatible with most major DAW software and file types

  • AVID PRO TOOLS
  • APPLE LOGIC
  • ABLETON LIVE
  • STEINBERG NUENDO
  • STEINBERG CUBASE
  • PROPELLERHEAD REASON
  • PROPELLERHEAD RECORD
  • PRESONUS STUDIO ONE
  • COCKOS REAPER
  • APPLE GARAGEBAND

Conforming - Part 2 - Reconform

In part one we covered the process of assembling location sound recordings according to the picture editors EDL’s (Edit Decision List). In this part we will investigate what happens when they change their mind and do another turnover.

Before you say “but the picture is locked” realise that this is not the case…ever.

In this case you will need at least one of two things. Either the Video EDL’s from both versions of the picture or a change note (also referred to as change list) between versions.

Change notes can be great for doing the change conform by hand, if a little confusing at first. Trim head by x frames at this footage, trim tail by x frames at this footage. The great thing about change notes is that with a little work they can be whittled down from the 122 events that it lists originally to something a bit more manageable by combining the events that occur at the same footage for an overall change. 

The downside of change notes is that they are susceptible to pollution by VFX updates and unless the picture assistants are very focussed in their naming/tracking puling these VFX shots out becomes not only a chore but an impossibility. Recently I’ve seen change lists 24 A4 pages long for the addition of 35 frames due to the VFX shots being listed individually as delete x frames, insert x frames. Not very helpful.

This is where the Video EDLs come in. 

Over the course of the last 10-12 years video EDLs have become more prevalent for change management than change notes are. This is for a number of reasons:

 

  1. They are easy to produce
  2. They are easy to manage
  3. They provide more information than change notes

 

The downside is that they are more difficult to read for a human due largely to the split line format, maybe that’s just me though.

Many programs have been developed to compare versions of an EDL and compile a change scheme from this comparison. The three main contenders in this space are: Conformalizer, EdiTrace and Virtual Katy.

Each of these applications compares one or more EDL’s from a previous version with one or more EDL’s from the current version and creates it’s own change plan from it, which it will then carry out on some DAW’s. The differences come in the feature sets available and this is where opinions become wildly different. 

EdiTrace by SoundsInSync

As you will know from looking through my previous posts I really like Mark Franken’s work on a number of bases EdiCue is superb and EdiLoad is without a doubt my favourite tool for doing anything with an EDL. Editrace is a little different in it’s approach to the comparison problem.

Basically you subscribe to the Editrace web site and from there add Projects and EDL’s via upload. When you need to do a conform you select the source EDL’s and a single Destination EDL and the web site does a comparison for you and creates a Change EDL for you to download. 

The resulting EDL is very clean and does a great job of weeding out VFX inserts and other extraneous information which makes for a pretty clean conform. This EDL is then loaded into the EdiTrace desktop application for auto-conforming in Pro Tools.

Conformalizer by Maggot Software

I have really wanted to like this application. It does some great things: QuickTime comparison to edit changes is a superb feature. I always felt it was limited in it’s project based features though and for me that was enough to break my interest in the application. For the odd one off conform it could be the tool for you though. Go to the web site and check it out. Justin is a very active member on the DUC too and answers questions and feature requests very quickly.

Virtual Katy by Virtual Katy

The feature set of VK scares me. That’s what I love about it though, if you work on shows with multiple versions then don’t buy anything else. It’s major strength is that it keeps track of all your EDL’s across an entire project. By loading all the EDL’s for all reels into a generation you can then compare across all reels/generations for any new cut that comes your way. 

This is great for: rebalancing, old material going back in much later on and when it goes wrong (through human error) cutting awesome trailers ;).

As of the latest version it has gained a RegEx (Regular Expression) engine too. RegEx is a method of matching text strings against a set of rules in order to replace/remove those strings. Why is that good for conforming? Well if you have a show with lots of VFX then you be familiar with EDLs that contain strings such as: “76-5 V_dNeg_72638”. What RegEx can do is strip out the data that is likely to change in the next versions EDL ie 72638 might become 72647 and leave you with the slate and take info. So instead of matching “76-5 V_dNeg_72638” across versions the comparison engine is now looking to match 76-5 across versions. 

This feature allows more material to be saved and in the process fewer cuts are made and your sessions look less like they’ve been through a shredder. 

You want material from Temp 1 to go back into Temp 5? Easy just choose Temp 1’s generation from the list and compare it with Temp 5’s generation and in seconds VK will compare all the shots listed in all the EDLs across all reels of both versions. From this list you might only want material from 02:10:15:23-02:12:57:13 to be retrieved. Easy, just select those Timecodes from the comparison list and hit conform selected. 

The problem that I’ve most frequently come across is getting picture assistants to provide all Video Layers to me as EDLs. When I haven’t had these is when the trailer cutting comes into play. Suddenly material from reel 5 will slot itself into reel 2’s boat chase. If the picture assistant still won’t give you the EDL’s you have asked for, just export the QuickTime you’ve just cut together from the timeline and they’ll have them to you very quickly after that. 

Conforming - Part 1 - Assembly

Sound post production is a confusing field at the best of times with the pull-up and pull-downs, sample and frame rates and all other sort of things that have their own names. Then over the years someone decided that calling two entirely different processes by the same name was the best idea since the NLE.

So what do these two terms mean and what tools are around to help with these processes?

Conforming - Stage 1

The first process that is named conforming is also referred to as the Assembly. I think I am right in suggesting that in the US they use assembly more than conform for this process but I’d imagine it varies between companies even then. 

This process involves the construction of traditionally split channel audio from the mix downs that were used by picture editors to edit with. Why is this necessary?

On set the Floor Mixer who is often referred to as the Sound Mixer (different from re-recording mixer) on credit rolls and the like will take the various channels that are being recorded (for the sake of argument lets say 8 discrete channels) and create a rough mix of these channels. All recordings on set contain a timecode reference that is linked with the cameras on set. Once the shoot has finished all of the footage is loaded into the picture editing system together with the sound recordings (sound rolls) and either auto-synced or manually synced by the picture assistants. 

The mix track created by the sound mixer will be used on track A1 and A2 of the editing system as track limitations exist in all of the picture editing platforms. (they have increased in recent years but they still have limits)

Once a picture has been finished to a decided milestone then the picture editor will turn this picture over to their assistant to provide to the sound crew. 

A standard turnover for a feature project will contain: Picture, Audio Guide Tracks, OMF/AAF (Open Media Framework/Advanced Authoring Format) which presents the picture editors sound tracklay and EDL’s (Edit Decision List) both audio and video versions.

(It should be noted that while change lists are often provided to help with the second version of the conforming name they have become less reliable with the dawn of visual fx drop ins etc and are often so polluted that they are unusable)

After all that we reach the conforming (meaning 1) or the Assembly. A sound assistant will take the Audio EDL that was provided and either by hand (very rare) or using an automated program construct an expanded version of the OMF/AAF for use by Dialogue editors (primarily). 

The EDL contains references to the Tape used (sound roll) and the Timecode start and end. 

Traditionally this process would have involved recording the relevant audio into the edit system from DAT tapes etc, but this has now mostly changed with the advent of hard drive based recorders. More often than not the assistant will be referring to a list of folders on a drive. In order to perform this process manually they would refer to the sound report sheets for the start/end TC’s of the files. 

Thankfully programs exist that do this leg work for you. The most frequently used is Titan by Synchro Arts, as of version 4 this is now capable of writing/reading to and from Pro Tools version 7 format files as well as to the AES31 format.

Useful Programs to help

Ediload - Great application for tidying up EDL’s. Discipline in naming of sound rolls appears to be becoming a thing of the past but this really helps sort those issues. Batch editing of sound roll names etc.

Titan - The only application to consider for large scale assembly projects

Alternative/Up and coming methods

As of version 7.2 of Pro Tools, Avid have been including a feature set labelled “Field Recorder Workflows”. The feature set is now getting quite deep and significant improvements have been made to the way in which Pro Tools selects alternate channels etc. On projects with up to 15 sound rolls then you should give it a look. There a some prerequisites that are all listed within the documentation as to the best practices that should be observed but as long as sufficient metadata is being kept then you should have some degree of success. Unfortunately as your sound roll count increases you may find it difficult to maintain performance levels with this path.

Part two will cover the other process referred to as Conforming.

Any questions/comments please add them in the comments or drop me an email via the link on the left.

Mix Prep

Having spent much of the last 3 days building sessions ready for the mix stage I thought I’d share some of my workflow tips. 

IO (Input - Output)

When building sessions you really need to have enough IO to allow all of the tracks to be routed accordingly. For this show the sessions aren’t too wide so I’ve been making do with a 96, 2 analog 192’s and one Digital 192. This allows 64 outputs from a Pro Tools rig. 

It is possible to build the sessions without this capability but it takes much longer. Pro Tools has useful shortcut keys (shift+alt+command) for rippling outputs etc that only work if all the outputs are active. 

Before building the IO setup for a show I generate a spreadsheet that gives a representation of the systems so the mix techs can build the desk IO to match.

One of my greatest wishlist items for Pro Tools is offline IO setup. Meaning that I’d like to be able to route to ‘virtual’ outputs which would mean not having to connect a whole bunch of extra boxes to my system for this purpose.

Session Template

Next stage after the IO is building session templates for the various playback machines. I tend to do this step on the mix stage itself as then I can test the output routing against the inputs to the desk. 

Safest method for ensuring compatibility with the editors sessions is to take one of them (session not editor) and clear out all of the media from the tracks, import the IO settings and route the tracks accordingly. Many editors now route their tracks to busses and then output from the busses which makes building sessions harder and easier in equal measure but for different reasons. 

Easier - Don’t have to worry about the tracks just the busses

Harder - Output checking is either in busses only or requires some automation manipulation to test all outputs in order. 

When building these sessions I tend to create a test tone file of 1 second and 6 frames. 1kHz tone for the first second of the file then silence for the next 6 frames. I then copy and paste this down the tracks offsetting head to tail on each track. 

Once this is done I provide this session file and the test tone file to the mix stage. Running through the first section then allows all outputs to be checked for routing/functionality etc and any dead outputs to be identified and offset. 

It is important when building the session template that all possible tracks are built. For example if R1’s OMF only has 8 tracks but R5 has 14, then the session template needs 14 tracks built and routed. 

Start Times

Session template is set to start at 00:58:00:00 which leaves 2 minutes before the Picture Start mark on most projects (this will differ depending on whether it’s a NTSC or PAL project). This is also useful later on when it comes to combining reels or incorporating stems.

I leave the test tones in the sessions as if there is a problem it makes it much easier to find than trying to read the output label on all the tracks. 

As this is a template session I name it something innocuous and leave the reel as Rx. 

I then place this in the project folder of each reel and proceed.

Duplicate the editing session

Before I do anything else to the editing session I always create a copy. A quick cmd+d will save a lot of trouble down the line. Only ever work with a copy of the original session!!

Prepare the edited session

After opening the copy of the edit session I go to the picture start mark select all tracks and hit shift+return, which selects everything before the start of the picture. Delete anything that is highlighted. 

Next is to set the session start time. I had gotten into the habit of setting the session start time to match the picture start time (01:00:00:00 for example) as up until Pro Tools version 8 bad things could happen at the next stage, luckily this has been resolved so as long as the session start time doesn’t precede the start time of the template (00:58:00:00) then you’re good to go. Quit the Edit session and open up the template.

Once the template is open set the session start time to 2 minutes before the relevant hour. (01:58:00:00 for Reel 2 for example) this will push the test tones to that timecode.

Import Session Data

This is where the template comes into it’s own. What we have done is create a pre routed, standardly named file with full IO setup, correct session start times/formats etc. 

Importing session data from the copy of the edited file brings up the window to the left. 

Quick run through:
Top left shows the details of the session you are importing
Mid left deals with audio and video media to copy/link/consoldiate
Next section is the tracks contained within the session you are importing
Bottom left shows other elements of the session. Markers etc.

options on top right - assuming you have followed through the instructions above should never need to be changed for this process. 

 

Above the track listings section there is button called Match Tracks. Hitting this button should go through the track list and match each track to one in the template session.

Assuming that everything matches up (which it will do apart from the odd track) the most important box on the entire page is the one that says Some.

And heres what you need to select in that box.

Most of these setting are fairly self explanatory. Essentially what you are doing is importing the editors work into a mixer friendly format.

What you are unchecking explained:

Alternate playlists are a music feature primarily although I’ve seen some people use them for ADR.

Main Output Assignments - You’ve already done the routing in the template session so not selecting this ensures that your tracks output correctly

Voice Assignments - I uncheck it because mostly our sessions are dynamic voicing and leaving it checked could result in crashes while the system tries to reassign them. Unnecessary!

Input assignments - In the template I tend to set inputs only on tracks that require them: bus returns etc. Unchecking this means that the IO is kept clean and free from editor clutter.

I/O labels - Having spent time setting up my IO labels to match the spreadsheet document having them replaced by the settings on the editors rigs is just annoying and messy. 

Track Comments - Contentious issue this one. Most of the time I’ll uncheck it for the same reason as the Voice Assignments but occasionally if the editors have made use of them to eg/ clone the track names then I’ll reselect this box. 

Between the track listing and the OK button there are 3 radio buttons. Replace existing, overlay and do not import. Select overlay.

Once these options are set it’s time to press OK.

Bad Things?

Until Pro Tools version 8, if you imported a session that was set to the same session start time but was blank before the hour but set the replace existing material rather than overlay existing material then it would wipe out anything that was on those tracks regardless of whether there was material or not. 

Now it will only overwrite material that is after the first event of the imported session. Leaving our test tones in place either way. It is good practice however to overlay as the bug does exist in fairly recent releases that many people are actively using. 

And Finish

Final step in the building process is to check that the media that the session references is something that will be copied to the playback system. Alt+O brings up the Project Browser window. The Audio Files folder that you see here is a virtual representation of the Audio files referenced by the session, NOT the actual audio files folder.

Sorting by path enables me to see quickly if there are odd references (old OMF’s, Guides or something just on the root of the system drive) and then Copy and Relink those weird references. 

Once this is done, moving the session to the stage is just a case of copying the Audio Files folder, the template’d session file and any other relevant media to the stage machines. 

If all goes according to plan then when the session is opened on the mix stage, all media should be found, the tracks should be routed correctly, the plugins assigned, automation enabled. cmd+J and we’re good to go…

Picture Codecs

Inspired by : http://www.elementaltechnologies.com/blog/primer-inter-vs-intra-frame-video-compression

What is the best picture codec for use in Pro Tools?

Many picture houses try to supply us with H.264 codec as this ‘looks great’ and has small file sizes. While this might be true, working with it in Pro Tools becomes a nightmare. Scrubbing fails to operate smoothly, moving round the timeline is positively glacial and altogether it’s more hassle than it’s worth. 

As stated in the above blog post Intra frame images are single snapshots of each frame of the movie. This means that for each second of picture there will be 24 frames of video (assuming a 24frame project, replace this figure with 29.97, 30, 25, 23.976 as appropriate).

Why is this good?

This is good because the computer only has to process that frame. It doesn’t need to think about frame 10 if it’s on frame 23 meaning that scrubbing, timeline jumping… everything about the operation of the DAW is smoother. 

The various QuickTime codecs that are available at this point become reasonably similar, certainly in terms of processing requirements. 

As a company we have tended to stick with DV based codecs for the vast majority of our projects. As stated in a previous entry we use MPEG Streamclip for converting the picture if a picture department refuse/are not able to produce a DV codec for us. The most common codec thast we have been using recently is the DVC PRO NTSC. This is a DV based NTSC (29.97) codec that is well suited for the 23.976 workflows that have been prolific over the past few years. 

In the US they tend to work with Photo JPEG files. These have a higher picture clarity due to each frame being a JPEG image of the video footage. There is a trade off here with file size. a DVC file of 20 minutes comes to around 1.6GiB while a PhotoJPEG comes in at more like 2.8GiB. It’s not a massive difference but it adds up over the course of reels and 15-20 versions (excluding VFX updates).

What else is there?

In October of 2009 Avid released a new codec for QuickTime that has yielded some impressive results in testing. It’s called the Avid DNXHD Codec for QuickTime and it is wonderful. In my opinion it looks much better than the H.264 equivalent and it has NO additional overhead to a system. 

A standard photo JPEG/DVC Pro QuickTime requires 8-10% of a CPU to playback within Pro Tools. The Avid DNXHD codec for QuickTime plays back with maximum (in testing) 8% CPU usage. 

For the next show that we do I’m going to try to specify this as our preferred option as it is closer to the playback specification for the stage. It’s only downside is that files are larger. 20 minute reel will be in the region of 4.5GiB. I figure we’l just keep fewer versions ;)

Omnifocus - How I love thee.

GTD - Getting Things Done.

I rely on Omnifocus by the Omnigroup to manage pretty much everything I do. From the weekly shop to financial tracking. Within a work context however it has become almost indispensable. Large portions of my job require repeating processes over and over using multiple programs, processes and often computers. Omnifocus keeps me on track. 

Turnovers

I keep a set of actions in an on-hold project called Utility. This has a fairly averasge set of actions that I perform over and over regardless of what project I might be on. Turnover Utility Actions

As basic as this action set is without it I’d frequently forget to do the Fx conform of Reel 4. 

Some of the actions are not relevant to every project. For example, on most projects the codec that is requested from a picture department is what we actually receive. So the picture convert part is irrelevant. 

For the record when doing picture conversions the only tool I now trust is MPEG Streamclip it is fast, can deal with batches and reliably outputs the correct codec at the correct frame-rate. QuickTime player on the other hand refuses to output a 23.976fps picture from H264.

Contexts 

Still get really get a handle on these but kind of make it work. The best thing about Omnifocus for iphone (you knew it would have one) is within the contexts you can perform Business Searches. So for example my shopping list has a context of Groceries; within that context it searches for supermarkets. If I ave some time I can check to see what actions are available and as part of it I get a map of the area with all the supermarkets highlighted, pretty neat. I think so anyway. 

All of the mix stages we use also have contetxs so if I need to move files or setup a mix session then I assign that to the action and it searches for Goldcrest/Pepper and tells me what needs to be done. 

It’s my memory Cap’n she can nay take it

Progress bars tend to be a constant part of my day. I set this encoding, that downloading, another thing uploading to a playback system or the server, titan conforming or fixing. Without my beloved omnifocus I’d quickly lose track of where I was at. Did I put the final PB session for Reel 5 onto the FX machine? Quick check, Yes I did. Phew. 

So there it is, my ode to omnifocus. It’s more than I ever need it to be but for what I need it to do it excels. Omnigroup also make some other wonderful programs such as Omniplan (Project Management), Omniweb (where my love for the company started but I’ve had to leave behind :( ) and many others that I can’t really place in my workflows. Thank you Omni!

 

Weekend Working: Mix Prep

So, it’s Saturday and I’m in work.

Dealt with the 1 reel of new material that came in the morning and now waiting on maybe 2 more before the end of the day. Read all the forums I follow these days, fixed a session issue for Lon, updated mix documentation (twice)….and now more waiting.

Spent some time in the theatre yesterday building some template sessions ready for importing the master sessions into. This way you don’t risk losing automation by changing the outputs on tracks after import and you don’t end up duplicating paths in the I/O setup window.

Only problem I have now though is that my Pro Tools rig doesn’t have the necessary plug-ins or processing cards available to build the sessions for the stage. Not really sure how to get around that at the moment but I’m sure I’ll figure it out.

Got a couple of hours to kill today and a whole heap more tomorrow. Might end up waiting to build the playback sessions till tomorrow when there may be a system available which has the requisite cards/plugs available.

Keeping track of the various machines media copies using Omnigroup’s marvellous OmniFocus application. Great for GTD management, and has an iphone app that updates via MobileMe.

Have fun.

Ceri