Camera Club AI for video production

- Written by Hamish Beeston. TL;DR - bad sound clips magically fixed. Before and after transformations below!

Last week we held our latest ‘Camera Club’ session at our offices in Hamilton House, Stokes Croft.

The Beeston Media team were joined by fellow Hamilton House residents and filmmakers Under the Wing Productions and we looked at examples of how AI is streamlining our workflows and processes from pre-production through to post-production. It was fascinating to compare the different tools we are all using and how AI is affecting our production lives - for good and ill. Thanks to our own Jake Michael Christian for researching and running the session.

Stand out tools include VEED.IO for prompt-based photo realistic video generation, story-boards.ai for prompt-based storyboard creation, complete with video made from the storyboard at the end. Gemini, which is our office regular LLM, is great for prompt-based research and also first draft scripting and Google Earth’s ability to check out remote filming locations and crucially, the angle and height of light at a given time will be most useful.

For me, the area in which AI is having the most immediate impact is in sound post-production. The received wisdom in video / film is that viewers will forgive poor imagery / camera work etc if the content is interesting but that poor sound is a disaster and makes a clip unwatchable.

There can be many contributing factors to the quality of recorded sound, some in your control such as the locations you choose to film, the type / position / quality of the right microphones for the conditions and the recording levels. Other factors are often beyond your control like having to take a feed from a dodgy third party PA system at an event, excessive and unavoidable background noise and random radio mic interference.

The most memorable sound catastrophe from my long career in production was when I made documentaries for the BBC and a sound recordist turned to me, the director, at the end of filming a whole evening performance of a show to tell me there had been ‘air in the line’ i.e. he didn’t plug into the back of the camera. He’d been monitoring the lovely levels on his sound mixer oblivious to the fact that nothing was going down on to tape. Talk of tape shows how long ago this was but even in those more basic 4:3 Betacam days, you still had to remember to plug a cable in!

Luckily there was another performance the following night and the crew shot it all again - for free - from complementary angles so the final programme coverage was amazing. They also made sure that the sound feed was connected to the camera this time.

We’ve used Adobe Podcast Enhance for a year or so to improve badly recorded sound and V1 has been great - but with limits. As with many noise reduction programmes, if the voice is a similar tone to the background it can struggle, leaving the output strangely thin and inauthentic. V2 however is a massive step up. With separate voice and background sliders, you can isolate the elements and crucially the AI is able to recreate missing sound. It also manages to reinstate an authentic shape to the vocal, even when missing from the original.

To illustrate quite how game-changing AI technology is for sound, here are some before / after examples we discussed at Camera Club:

1. Echo

Filming in a room with an echo is a common problem, even in not particularly voluminous spaces. The example below was shot in a standard event venue reception room which crucially has no soft furnishings at all. Pre-AI, echo was one of the hardest things to fix and one of things clients noticed most. No longer.

2. Wind

Wind jammers work well to enable recording sound outside - up to a point. On this occasion, the fluffy mic cover reached its limit as the wind peaked and the clip was ruined. Or so we thought until we tried AI.

3. Interference

Radio mics are amazing, especially the new cheap and excellent clip-ons that you see all over TikTok. Until that is, they’re hit by random interference. We recorded 4x talks on this particular day using these radio mics. 3x were perfect. The other one wasn’t. Until AI saved the day.

4. Distortion

Arguably the above examples weren’t really anyone’s fault, simply bad luck. But sometimes, pure operator error is the cause of bad sound. In this case, audio levels were just way too high. The audio was bad in the room and bad on the feed, leading to a justifiably unhappy client. Today, with AI, it would have all been fine.

The future? Get your interviewee to read a script which covers all possible dialogue combinations, like in Mission Impossible III where Tom Cruise takes Philip Seymour Hoffman’s voice print. From this recorded dialogue, we could scan the clients face and let AI create an interview...a future that to some extent that’s already here with applications like HeyGen, but we’ve yet to persuade a client to give it a go