Putting Lenses to A Taste Test

Seeking audience input? Denis Lenoir, ASC, AFC, ASK details his approach to conducting an audience-driven survey of spherical and anamorphic lenses.

During the recent Camerimage International Film Festival held in Torun, Poland, Denis Lenoir, ASC, AFC, ASK held two sessions to screen a lens test he put together with three colleagues: Caroline Champetier, AFC; Pascal Lagriffoul, AFC and Martin Roux. The test covered 33 pairs of spherical and anamorphic lenses. Viewers graded each in a blind format, without knowing names or brands. And rather than compare test patterns or gray cards, the test was filmed on a soundstage, with actors, props, and different light sources.

When introducing the test to the assembled Camerimage audience, Lenoir explained its background:

A few years ago I was prepping a film. I needed fast lenses because it would be on film at 400 ASA. I needed forgiving lenses because I had an older actress and my director didn't want me to use diffusion at all. So I decided to compare a few possibilities. I also decided to do a blind test, without knowing which brand was which. There were five of us in the screening to look at the test: myself, two assistants, and two staff members from the rental house. The five of us were astonished by the results. They were exactly the opposite of what we expected.

That gave me the idea to launch a larger comparative test. A year-and-a-half ago I coopted three of my French colleagues: Caroline Champetier, Pascal Lagriffoul and Martin Roux. We discussed for quite a while how we would build and shoot the tests. Pascal teaches at the École nationale supérieure Louis-Lumière, so we used a stage there. 

“This is not scientific, it’s your appreciation, your opinion of the lenses which counts. One of the benefits of the test is that you may discover a lens you had forgotten about, or didn't like previously.”

We had the association of production designers in France, the ADC, help us build the set. We also had the help of the association of focus pullers and camera assistants, the AOA. Over two days we tested 33 pairs of lenses. We actually filmed a little more, including zoom lenses, but since they were from a single brand, and I wanted to be fair to all the manufacturers and rental houses, we didn't include them here. There is also a lens I love from Panavision we decided not to include as it is large-format, and all the lenses in this test are standard-format.

Two-thirds of the lenses are spherical, one-third anamorphic, presented in random order. We filmed all the tests in a 2:1 ratio, which means the amount of pixels is more or less the same. For spherical, we shot 25mm and 50mm lenses, which would roughly be 50mm and 100mm for anamorphic. Some brands of lenses come in sizes like 55mm and 110mm, so the anamorphic may not be an exact match.

We used an Alexa at T2.8 and 2K. We shot a chart at the beginning of every shot, and the only grading we did in post was to balance each lens on a grey card. But these are not included in what we are screening for you. 

This is not scientific, it's your appreciation, your opinion of the lenses which counts. One of the benefits of the test is that you may discover a lens you had forgotten about, or didn't like previously. Or the test may confirm your opinion about a lens. The other aspect of the test that may be useful is that it gives you the chance to clarify your taste without being influenced by brand names and advertising. You may also be inspired to conduct your own tests.

You will be looking at a series of new images that are more or less the same. It's tiring, it requires a lot of focus. So be mentally ready for that.  

After each of the 33 test shots, the viewers at Camerimage had seven seconds to fill out a scorecard. Lenoir suggested that viewers might want “to grade on a scale of one to five rather than just writing ‘good’ or ‘bad,’ which may not be precise enough to use later on.”

The test repeated the first five examples to give viewers time to adjust. It took a while to train your vision, to decide what you found interesting in the image, and what information you would use to compare lenses. After the first round, shorter edits of the shots were repeated so viewers could judge a “Soft / Harsh” category. A third pass of edited shots were to determine “Flare.” Finally, the 33 full shots were repeated in a different order. As Lenoir explained, “When you originally compare lenses, your judgment may depend somewhat on the lens you've seen just prior.”

At the end of the test, viewers received “cheat sheets” to decode the names of each lens. 

Lenoir later spoke with AC:

American Cinematographer: When you were building the set for this shoot, what specific elements did you want to include? 

Denis Lenoir, ASC, AFC, ASK: To test the lens, you want to see how they perform, you want to see how they behave when you pull focus very close, you want to see the way the focus falls off, obviously you want to see the flare, you want to see the bokeh. The reason why we designed the set this way, and not like the many tests which are patterns and things like that, is that we wanted it to be like a film setup, with actors moving in a space, with windows and lights, racking focus, things like that.

We started by going to the space, a soundstage but one not immensely big, at École Louis Lumière and imagining where we would put the camera. Once we set a camera with a 25mm lens, we moved a few flats around. We worked with a production designer, Olivier Barthélémy, who helped us decide, okay, we will put a window here. We knew we wanted depth, so we put a doorway with a table and lamp behind it. We have raking light from a window we don't see, we also have the other window. The usual practical lights, but at different depths. 

Barthélémy was inspired by a photo book about Cuban interiors, hence the distressed look which we love because it provided texture. 

We had the actors, they were amazing, able to reproduce their actions so consistently. We were so lucky with them. 

You didn't want to work with technical committees.

I didn’t want to go through committees because it would be too complicated; there would be too many voices. Even among the four of us, I had to yield. Originally I wanted to separate the spherical and anamorphic lenses, because in my mind, they’re like two different domains. You took one or the other. But Martin, who is much younger, said, “No, it’s not like that anymore. We use anamorphic just for the look, so they should be mixed into the test.” 

What were some of the other issues you debated?

We had a lot of discussion about which stop to shoot at. The higher the stop, the better the lens behaves. I was afraid if we shot at too high a stop, all the lenses would look the same, great. And also myself I'm the kind of DP who works more at a 2, 2.8 split. 

Ultimately we shot everything at 2.8 for spherical and 4.4 for anamorphic. We used those stops because not every spherical lens can open to the same wide stop. Sometimes the widest will be 1.4, 2, 2.5. It made more sense to streamline everything.

There were a lot of passionate discussions, so with more than four participants I don't think it would have been possible to agree on anything! 

At one point the actor holds a paintbrush at the bottom of the frame. Sometimes you can see the brush reflecting a light, sometimes not. 

Everyone can pick what is important to look at in the frame. And what you consider important might change. When I watched this time, I saw things I never noticed before. With the paintbrush, I'm interested in the way the focus changes from the actress in the background to the paintbrush in the foreground, and then whether flare appears on the paintbrush or not. 

How do personal preferences affect the test? 

There's no right or wrong. With this test I'm not trying to say what brand or what type of lens is better than another. It's about your own taste, first what you like, then eventually what you would pick for a particular project.

Personally, outside of the test, before we even designed it, my own taste was for a softer lens, but no flare. I prefer to add flare with a filter in the camera or in post. But this is me. Younger DPs seem to be looking for “more flare,” “vintage lenses,” “let’s remove the coating.”

The viewers responded audibly to a couple of the lenses.

It's not because they are bad lenses. People may be using them in commercials or music videos and loving them. There's one that has the word “flare” in its name. Probably Dan Sasaki [VP of Optical Engineering at Panavision and an ASC associate member] was having fun, removing any coating he could find on any spot on this lens. He decided, “They want flare, I’m going to give them flare.” 

At one point I was thinking about collecting the answers and compiling statistics about the results to find out what is more popular. Then I thought I might create come problems, because for one thing these are our sponsors! But ultimately that’s not the goal of the test. I have strong ideas about what I like or like less, but I'm not trying to convince anyone. I'm not trying to give a medal to the best lens in the world, I’m trying to educate myself and anyone else who is interested. 

Educate in the sense that you have to learn what your own taste is. There are trends, fads, but maybe your taste is not exactly what you think it is. With this test, without worrying what brand it is, you can see how high you score a lens with a lot of flare, how high you score a lens that is more contrasty or whatever, and that way learn what your taste is.

I showed this a few weeks ago for the AFC, a screening Saturday morning outside Paris in the screening room of a rental house, TSF. Eric Gautier, AFC told me, “I was almost not going to come, because I was thinking, ‘Oh God, another lens test.’” And then afterwards he was extremely happy, because he was discovering lenses he had forgotten about, or lenses that he thought he was prejudiced against, and then he was actually saying, “I like that, I'm going to try that.”

So the feedback has been good, people are very happy. My goal was to start with the AFC, because I'm a founding member. So, for sentimental reasons the AFC first, then Camerimage because I owe them — they invite me every year! I'm happy to be able to give back something to them. I've been asked to show it in Belgium, to bring it to the BSC. I would like to screen it at the ASC and the Academy.

My message is: discover your own taste, discover what lens you like and don't like, and third, whatever you test the next time, blind is the way to go.

Some cinematographers don't have that luxury.

Yes they do. If they are deciding between the Alexa or the Red for some project, shoot them both side by side and look at the results without knowing which is which. Maybe you will choose the one you thought you didn't like.

---

The AFC previously reported on this Camerimage event here.