THE DIGITAL DILEMMA 2

ONE

On January 18, 2012, the silent film Wings, directed by William Wellman, was screened in the Academy’s Goldwyn Theater in a new restoration, with color tinting duplicating the original Handschiegl process, and featuring live organ accompaniment by Clark Wilson. But this was in no way a recreation of the original release. In 1927, Wings was heralded for its lustrous 35mm. imagery and for thrilling aerial stunts done in a pre-rear screen era, every frame captured as real events on nitrate negative film. This Academy screening, however, was made from a DCP, a digital file, and shown on the Academy’s 2 K Barco digital projector. Like many of the treasures of the silent and early sound eras, the original negative is long lost; the restoration was made from the Cinemateque Française fifth generation print using state of the art digital technology to restore (as far as possible) the shimmering glow of the black and white nitrate film original.

Several weeks after this screening, the Sci-Tech Council of the Academy released The Digital Dilemma 2 report, the long awaited follow-up to its 2007 paper that discusses preservation and archiving in the digital age. That original paper explained challenges confronting the major studios; the TDD 2 paper focuses more narrowly on indie and independent movies as well as on audiovisual archives.

Motion picture preservation is no longer the concern only of studio librarians, museums, and professional archivists. As we rush headlong into the era of digital cinema, archiving and preservation issues become more urgent, and need to be addressed even before a film concludes its brief theatrical or television run. In the world of low-budget indie and documentary filmmaking that is the focus of the TDD 2 report, these responsibilities now fall mainly into the arms of the filmmakers themselves, and unfortunately, there is often no money available for them to enlist professional, outsourced archiving. Discussing this past summer’s Academy screening series of documentaries, one staffer confessed to me that some documentary filmmakers keep their digital masters in their own homes and, in one case, under his bed.

The first century of movies was photographed on film: first on fragile, flammable nitrate base stock and by 1951, on acetate safety base. The mantra for storing original elements such as black and white and color negatives, 3 strip Technicolor matrices, and reversal masters has been “store and ignore.” This may be a too reductive aphorism but it does define the lack of concern, even the confidence, with which motion picture film elements have been stored in sometimes problematic film vaults, vaults often built more for fire protection than for the critical control factors of temperature and humidity.

In June of last year, David Pierce wrote a report commissioned by the National Film Preservation Board titled, Survey of the Survival of American Silent Feature Films: 1912-1930. Despite its unwieldy title, the report is a fascinating look at the history of the American Silent Film with its attendant erratic distribution patterns. It tracks the many differing gauges and formats both in original production and in downstream distribution and exhibition. The discussion of the many different versions of a film, its print formats and preservation masters reads like a jigsaw that bears more than a token resemblance to that of ever increasing iterations of digital video media, formats, and codecs, a maelstrom that is swamping us. Any cinematographer bold enough to buy a high-end video camera today must expect to profit on its rental within a several year life span—before the camera’s inevitable obsolescence.

A look at the number of silent American films that have survived until today may provide a cautionary tale for filmmakers in our digital age. There are 1,538 American silent films that exist in complete 35 mm. domestic versions. That may seem like a lot until you realize it is only 14% of those that were produced; another 1,090 (10%) are complete but exist in only lesser formats or foreign release versions. (It was common practice for multiple cameras to provide several negatives for international distribution, taken simultaneously at different camera angles and with slightly different shot sizes.) Less than one fourth of all American silent motion pictures exist today in any complete form—and they were all photographed on the medium that is still universally considered to be the only archival one—film. To be sure, there was little thought in the pre-TV, pre-home video age that there would be anything like a “long tail” life for these movies. Films were largely disposable. Several studios emptied their silent film vaults in the thirties, reclaiming the valuable, dense deposits of silver that were embedded in the nitrate negatives and prints.

These survival statistics, though, come from the “store and ignore” mindset. If those early movies had been stored on digital media, it is certain that nothing would have survived. When many of us were film students, older movies, especially documentaries and foreign films, were mostly available for study on badly duped and scratched 16mm. prints. It is one of those wonderful ironies that it is by the magic of state of the art digital technology that today the masterpieces of early cinema are being given anew to us in pristine digital editions. “Wings” is such an example. Several months ago, TCM broadcast this movie as well as glorious digital restorations of the 1921 Four Horsemen of the Apocalypse and the 1925 Ben-Hur.

Three strip Technicolor films such as The Wizard of Oz and The Red Shoes, with their unevenly shrunk matrices, have most benefitted from the tools of digital restoration and have allowed us to experience anew the glories of Jack Cardiff's cinematography.

Like its earlier model, The Digital Dilemma 2 report is well structured and very readable, devoid of cant and techno-babble. It is clearly intended for working film professionals in many disciplines, not just for archive moles and equipment nerds. In separate chapters, the report deals with independent filmmakers, documentarians, film festivals (marketing and distribution), and in-depth surveys by several of the eighteen audiovisual archives that participated in the study. There is an introduction and brief history for each section. An eight page appendix features an interview with Peter Mavromates, a director of post-production, who has worked with many studios and filmmakers, notably with David Fincher on several of his “born digital” feature films, as well as on the recent Iron Man 2. Mavromates’ perspective paints a confident picture of an eventual all digital future, but for now at least, a cautious one. When asked, “What do you do over the long term with the digital files (for a feature film),” he answers, “If you are asking is the film-out an interim solution—no. It is a delivery requirement and although the film-out is certainly an archival element, we consider the DI the mother lode. For both now and later, the solution is data migration.” This may well be true, but the problem for many digital filmmakers who do not have the unlimited resources available to Mr. Mavromates’ studio clients or the deep pockets of well-endowed audiovisual archives, data migration on a regular basis is a pipe dream.

Just how precarious the preservation/migration issue is for indie filmmakers and documentarians is reflected in a Jan. 18 article in Variety written by David S. Cohen. He interviewed Milt Shefter, Sci-Tech Council member, who co-authored the report with Andy Maltz, Academy Director of the Council.

Shefter told Variety, ”The bottom line is we’re running out of time. The time for studies is past. We have to find solutions or we’re going to lose a lot of material.” For documentarians this ticking clock is critical because it is not at all unusual for their films to achieve distribution only after the “shelf life” of the non-migrated master has passed. Even worse, according to page 4 of the report’s executive summary, “Most of the filmmakers surveyed and interviewed… were not aware of the perishable nature of digital content, or how short its unmanaged lifespan is.” Many documentarians depend on access to historical footage; they seemed equally unaware that news and documentary footage they may draw upon is just as vulnerable to being lost.

It is in our nature to believe that somehow one  technology will always ride to the rescue of another technology; that may not be the case here. The report’s executive summary paints the unvarnished reality as clearly as possible:

Well-established, time-tested analog preservation practices do not apply to digital holdings; digital materials are fundamentally different from motion picture film and other analog materials. Suitable long-term preservation and access mechanisms for digital motion picture materials have not yet been developed.  (my boldface)

And until they are developed every motion picture image that exists solely in the digital realm is at risk—just as are our personal iPhoto files on our laptops and iPhones… Have you lost any files or photos? “Future-proofing” in the digital world is just a slogan, not a reality.

One of the most intriguing sections of the report deals with nonprofit audiovisual archives. They are the real repositories of our A-V history and their custodians are equally alarmed. According to the National Film Preservation Board, there are 550 moving-image archives in the United States and its territories and another 310 worldwide. Preservation of film materials in the United States began early. According to TDD 2, an early version of “migration” of movies began in 1942 when “the Library of Congress discovered paper print copies of several thousand motion pictures produced between 1894 and 1912--- deteriorating in one of their vaults. The Library restored and converted approximately 3,000 of these paper prints to motion picture film through collaborations with the Academy of Motion Picture Arts and Sciences and Renovare" (a company formed for this specific project). Paper prints were originally made frame-by-frame from the OCN or fine grain master positives because there was not yet copyright protection law for film--- but there was for material created on paper. Some of the early Griffith films exist today only because of this measure.

One section of the report tries to explain the concept of “digital preservation." It includes this caveat from the United Kingdom’s Joint Information Systems Committee:

Digital objects will cease to be accessible without active management and intervention. The biggest risk to the accessibility of digital objects is the continual development of computing hardware and software. Many digital files or formats are dependent on a particular computing environment for accurate presentation of their content. Any change to the rendering environment could result in change to the rendered representation of a resource (or result in not being able to render the resource at all).

Once you dig into the dry jargon embedded in the statement, the message is clear. New systems and upgrades can exacerbate the problems they are trying to solve and even destroy content. Any of us who has done a home computer operating system upgrade, only to find that one or more applications no longer work, can understand the dilemma. It’s not unreasonable to say that the term “digital preservation” is an oxymoron. “Digital data management” is more accurate.

TWO

To put a human face on all this cascading information, I asked Milt Shefter if he would answer a few questions that came up for me as I read The Digital Dilemma 2 report. Milt is a good friend and an associate member of the American Society of Cinematographers. He speaks here as a film preservationist, not as an official Academy spokesman. Milt is also well known to many of us as the resonant voice of the annual ASC Awards show. Here are my (admittedly, sometimes leading) questions, with Milt’s responses:

JOHN BAILEY, ASC: Almost five years have passed between the first and second Digital Dilemma reports. Do you feel that filmmakers are more aware today of the challenges posed by archiving movies, television, and documentaries only on digital platforms?

MILT SHEFTER: No. And the indie filmmakers don't seem to realize that because of the time lag to distribution, their digital master may be technologically obsolete before that.

Why does there seem to be so little awareness and action by indie and documentary filmmakers to do long term preservation of their work? Is it simply lack of funding? Just a few years ago, YCM separations or IPs were routinely included as a line item in budgets.

That's two questions. One, we can't peer inside the minds of filmmakers. Many seem enamored with the perceived benefits of digital image capture and workflow in post-production. But, the lack of long term guaranteed access in digital storage technology is not considered. They're used to "store and ignore" analog (film) and may not understand that digital records require continuous and active management. Two, YCMs and IPs coming from film masters are economical. With a digital source master, you have to go through the Digital Intermediate master system, which is very expensive. Even just scanning back to a single strand (IP) film from a digital master is much more than most indie filmmakers can afford.

Are you surprised by the large number of filmmakers, esp. in indies and documentaries, who are still unaware of the vulnerability of work stored only as digital tape masters or on drives?

Yes, especially the documentarians who use archival footage for many of their projects. What will they use in 25 years when they want to use material from today, and most of it won't be in existence?

The major studios seem to be acutely aware of the archiving and preservation issues at hand. Have they reached any consensus about what to do? Do they have in sight any method or system other than media migration that is comparable to film for long-term preservation?

The majors take one version of their digital source master and scan back, usually to three-color [film] separations. That's their preservation master, good for the proverbial and proven 100-year life.

It has been over ten years since cinematographer Roger Deakins did one of the first DIs on a major theatrical film, O, Brother Where Art Thou? This exceeds the recommended 5-year "shelf life" for media before migration is necessary. We can trust or hope that this master DI has been migrated. But what about the perhaps hundreds of movies that have employed the DI process—and may not have such keen studio oversight, movies that failed at the box office or whose distribution arm has gone out of business? What fate awaits them?

Those that may have survived technological obsolescence are still faced with the issue that the DI facilities all used their own "secret sauce" software, so if those post houses are gone, even the surviving record needs "forensic" (read that as expensive) action to determine if the software (movie) can be salvaged.

We talk about "migration" as if it were a foolproof system, like a text clone; but in fact, the ones and zeros that constitute a digital record of a two-hour feature length film at 2K resolution, is a monstrously hungry data system. How accurate is such complex media migration? Has enough time passed, and enough migration generations, for us to know the safety and accuracy of the migration process?

Just the opposite. We already see issues with migrated materials. And, while it may work for a very small number of titles with active management, migration doesn't scale for larger collections: this from the Storage Industry Network Association study. The Associate Librarian of Congress, a major migrator of digital data, admits that each time they migrate, they face the danger of losing information.

Is it safe to assume that most migration is done on robot systems without a human eye as intermediary to check for accuracy? If migration errors happen, and they are almost inevitable, how does this affect subsequent migrations? The question of migration errors is not addressed in the TDD2 report. Perhaps, it's too early in migration history to even know the answer.

Our role was to ask about possible solutions for the long term. Migration, as stated by others (see above, Mavrotes) isn't a long-term solution. It's part of active management and has to be done before the five-year "guaranteed" life of the hard drive is over.

Given the importance of the original Digital Dilemma report, the Academy must hope for a greater awareness of the issues it addresses again in TDD 2. Yet, in general conversation, even among my cinematographer peers, I find much more interest in new cameras, LUTs, workflow, and DI systems than in any discussion about preservation and archiving. This dismays me, as we all seem, on the surface at least, to believe that the best of our work will outlive us. Do you have any idea why this lack of concern is so broad among those of us who are otherwise so techno-saavy?

As you know, TDD,2 was written from the perspective of the independent filmmakers and nonprofit audiovisual archives. The majors and other big organizations and businesses have resources and built-in ecosystems. The indies and archives are for the most part under-resourced.

As for the cinematographers, their concentration seems to be on the image capture systems and perhaps on some part of the post work flow (like timing.) It's sad to think that much of their creative efforts won't survive for future generations of audience or filmmakers.

How does the Academy hope to get the word out, especially to young and emerging filmmakers who are mainly digerati, that their digitally captured movies are a "slow fire," what has been called "digital nitrate"?

The Academy has committed to "get the word out" through film schools and film festivals to educate those filmmakers about the lack of long term guaranteed access in current digital systems. More importantly, it encourages the effort to try and organize collaborative activities, to get standardization and perhaps to push the envelope in creating an archival system that either meets or exceeds the parameters of the current analog (film) system of preservation.

THREE

“Getting the word out” becomes more critical by the month. The transition to digital production rushes at us now at a breakneck pace, despite the very real concerns discussed in this report. Near professional quality image capture is simply so easy with prosumer cameras. New and ever improving camera systems such as DSLRs are seductive, and with Arri’s breakthrough Alexa camera and the arrival of Sony’s F65, film negative quality in image capture is a near reality. It is so tempting to focus only on the technology we see. The speed of its ascendancy has surprised everyone. Momentum counts in movie production, just as it does in a sports event. A heady techno-fervor sweeps aside sober reality. That’s just the way it is. Motion picture film diehards, even those of us who have photographed many digital movies, are hard put to defend a “mature” analog system that reeks to the hipster's nose of a steampunk technology (film)—all those chemicals and cans, ugh. And the great, now beleaguered, company Kodak seems to be in a tailspin from the top, as if digital camera manufacturers had planted a mole to dismantle the company from within. The irony is that Kodak's Motion Picture Imaging division is one of its few bright spots; the long term (mis)management of  an iconic American company only unleashes more grief for our colleagues at Kodak Hollywood.

I doubt any of us knows just how all this will play out in the next several years.The dilemma of The Digital Dilemma 2 is just that--- a catalog of challenges seeking solutions. The new digital image generating systems that we embrace with such passion are also the sources of their own content demise: in short, unless we find solutions beyond the limits of our current technology—many filmmakers are facing a digital fadeout. Or a total blackout.

The Digital Dilemma 2 report is easily accessible at the Academy’s website as a downloadable PDF. Here’s the link:

Oscars.org: Science & Technology Council projects: Digital Dilemma 2

Get your copy today. It’s a necessary but disturbing read.

Next: The Three Graces, The Snapshot, and Roland Barthes

Please subscribe to the blog in the box below.

Tagged:

Comments

Subscribe Today

Act now to receive 12 issues of the award-winning AC magazine — the world’s finest cinematography resource.

Print Edition   Digital Edition
December 2017 AC Magazine Cover November 2017 AC Magazine Cover October 2017 AC Magazine Cover