JRR Tolkien's Lord of the Rings is a great yarn. Tolkien creates a world peopled by the most fantastic of creatures - wizards and dwarfs, orcs and elves, and nameless things from the bowels of the Earth.
In many surveys it emerges as the most popular book in the English language written during the 20th century. Precisely for that reason, it is a book film-makers have shied away from. Film imagery can never compete with a reader's imagination. Or can it?
At top, a computer model of the scene of a battle, with the terrain triangulated. At centre, live actors on a set. Note the marks on the blue wall behind them. At bottom, the actors are transported into the model.
In 1968, Stanley Kubrick's 2001, a Space Odyssey pioneered the computer control of camera position. Kubrick used computers to achieve repeatable camera movements in his photography of models of spaceships. George Lucas, in the Star Wars films, continued the trend. Lucas established a company, Industrial Light & Magic, especially to develop new techniques by which computers could be used to create special effects.
Out of that has come a whole new technology of computer graphics. It boasts that if anything can be imagined, it can be filmed. So it was only a matter of time before Tolkien's world and the new technology collided.
For the past three years, film-maker Peter Jackson and a team of technicians at Weta Digital in Wellington have been answering the challenge. After the release of the first of the three-part series, most film writers - and most film goers - agree that film-making will never be the same again.
So why should any of this be of interest to surveyors? It turns our that there is a problem lurking here that is of particular interest to the profession.
Computer graphics creates a world of its own. Programmers have developed many clever techniques to ensure that this world behaves as much like the real world as possible. The problem comes when the computer's world and the real world coincide.
So, for instance, standard animation packages now allow an aspiring computer artist to create a 3D world, to people this world with objects of some kind, to make the objects move through the world, and then to show the results from an arbitrary point of view.
Camera angles, and the placing of the actors, helped a lot with the illusions used in Lord of the Rings. In this scene, Ian McKellen, who plays Gandalf, sits much closer to the camera than Elijah Wood, who plays Frodo the Hobbit. In Tolkeinis world the wizards tower over all other folk, while Hobbits are small. McKellen and Wood are, in reality, quite normal. As in the scene on page 14, the background in this scene has been digitally added and morphed to maintain the illusion.
Today, this is pretty basic stuff. More advanced programs allow the user to script anatomically realistic 3D movement of people and animals. Given a certain amount of computer power, it is possible to drape an image over such models, or create surface textures. In so doing, it is possible to create imagery that is almost indistinguishable from the real thing.
No matter what the level, the important point is that such animations move through a proper 3D space. They are not 2D drawings.
This is OK as far as it goes, but often it is not far enough. Often, the film-maker wants to superimpose the two types of film-making. He wants to put a computer graphic of a tower in a real landscape, or make a real actor lean against a computer-generated wall.
Blending the images is, by itself, not difficult. Film-makers have had access to Chroma Key for years. In this technique, a foreground object is filmed against a blue background. The blue pixels can then be edited out and replaced with some other video output. The technique is regularly used in television to show video behind a newsreader, for instance.
But a news studio is essentially a 2D problem. If one requires the freedom to move the camera, or one needs to record the action in three dimensions, the real world and the computer's world need to be coincident in two ways. Firstly, the computer world in which the 3D model is created needs to be exactly the same size as the real world - the set - and secondly, the camera movement in both worlds also needs to be identical; creating the move in the virtual world is a process called match-moving.
If this can be arranged, the implication is that film and computer graphic worlds can be superimposed on one another, and can interact as the film director requires.
How to do this? There is a variety of different techniques that can be used for match-moving. Clearly, in many scenes, where the camera is stationary, it amounts to no more than being able to identify its position properly in relation to the rest of the set.
Tracking a moving camera is more difficult. Often, the easiest way to do it is to mount the camera on a motion-controlled camera rig, a kind of robot, and then move the camera by controlling the rig. The instructions to the rig can then be given to a virtual camera in the computer world so that it can mimic the real camera's movements precisely.
The adventurers scramble up a stairway deep in the mines of Moria. The actors are real; but the rest of what you see here was built in a computer.
However, there are a number of situations where motion control is not appropriate. Tight shots of actors moving or interacting often require a hand-held camera. In these instances, one of the easiest techniques for recovering the camera path is by using photogrammetry.
Small survey marks are placed around the set in order to obtain suitable geometry - these are removed digitally in post-production. Then the camera position on each frame can be calculated and once again, the path can be recovered.
However it's done, having established the camera path, the next step is to dimension the set accurately. In the past, this has been done with nothing more sophisticated than a measuring tape. But the scale of the Lord of the Rings film trilogy - millions of feet of film were shot and a good proportion of the film, when it arrives on screen, will contain some 3D element - meant a whole new level of sophistication was needed.
The commitment to surveying on the film was quite extraordinary. There were no less than three live action film crews working at any one time, and sometimes five. At least one crew was shooting miniatures, but sometimes there were up to three. To cover this there were nine surveyors. Up to four total stations were in use at any one time. In all, some 360,000 points were surveyed.
To add to the complexity of managing this process, locations were scattered all over New Zealand. Some were only 100 metres down the road from the Weta office in suburban Miramar, but others were at remote sites with helicopter-only access.
The survey team from Weta Digital, the company that designed the computer effects, investigated several different methods of dimensioning a set during the planning stages of the film. GPS was quickly dismissed - given the time constraints on a working set, not even RTK units can deliver the kind of accuracy required.
A number of pole-mounted surveying systems were considered, including one that used electronic distance measurement from several points. This gave very accurate results, but it was rejected because of its intrusive nature.
They also established that a laser scanner, such as those from I-Site or Cyrax, would solve the problem. Laser scanners are capable of generating an enormous number of points of the requisite accuracy - they can measure a point in three dimensions every few centimetres, generating many millions of points. But expense and lack of flexibility meant this option too was discarded.
In the end, they chose total stations - Leica 1103s and the motorised Leica 1105s supplied by Global Sciences in Auckland. They could be mounted off the set, out of the way. They deliver the required accuracy. They are also tough enough to handle life on location.
Models can be combined with live action and real landscapes provided both are dimensioned approprately.
So, with the total station off to one side, it was possible to obtain the dimensions of all the most significant parts of the set. The survey marks used for match-moving were pressed into service; they were matched against equivalent points in the digital model, ensuring that both worlds were coincident.
Often these marks were no more than pieces of fluorescent orange tape and the reflectors from pushbikes - they enable the total station to record ranges of up to 450 metres.
An additional layer of complexity was introduced into this process by the use of models. The model would sometimes be superimposed on the landscape; in some cases, it would be used as a backdrop to a set.
For set surveyors, this is an especially taxing problem, because the model is necessarily built in miniature. Saruman's tower, Orthanc, is one of the most important locations in the first film. It looks as if it projects several hundred metres into the sky. In fact, it is an eight-metre high model built in a studio. The model space is scaled precisely (in this case at 1:35) so that it can be placed in the landscape, and so that Sir Ian McKellen, who plays Gandalf, can be digitally placed on its topmost platform.
The problem here is that any errors in surveying the miniature are magnified by the scaling factor.
Surveyors working on the film stressed the need for accuracy during this process: 'We quickly learned,' said one of them, 'that the more accurate we were, the less rework would be required late in the production process to tweak the different layers in the finished film.' Of course working with a model is less stressful than working on the set. It was possible to set aside time to do the job properly, as opposed to working on the set, where the demands on camera, lighting and sound operators, not to mention the actors themselves, would take precedence over the need to dimension things properly.
By the end of filming, most of the crew were beginning to understand that giving us a few minutes on set to take a measurement actually saved hours of work at the other end of the process. But it took quite a while to get to that stage.'
The lesson was rammed home during production of a lengthy shot that involved the actors running down a long stairway. The actors ran down normal-sized blue stairs, but in the film they appear to be in the middle of a complex cave system. In fact, the interior of the cave is a miniature; each stair tread is about 10 mm deep. It proved quite impossible to measure them accurately with the total station. As a result, each step had to be manually inserted into the digital model.
One of the most interesting things about the new world of animation is the way it is changing the methodology of film makers. Until recently, film-making was a fairly anarchic affair. One started with a more-or-less detailed script, went out to the location and shot it more or less according to the script, and then took the results back to the editing studio. There, the director and the film editor would closet themselves for months, making decisions about which shots to use, how long each shot should run and how it should link to the next one.
Film maker Peter Jackson. His Lord of the Rings has redefined the art of the possible.
The use of computer graphics in a film imposes a much more rigorous structure on the film-making process. It begins with surveys of the location. Sets are then designed on screen, and the movements of the camera and the actors precisely determined.
From these plans, sets can be constructed. In the case of Lord of the Rings, AutoCAD was used for this. Then as-built drawings are constructed, and imported into the computer graphics package - in this case Maya, from Alias Wavefront.
As a result of this process, the entire film-making effort was much more rigidly planned than would normally be the case. There was less wasted footage and less wasted time on set.
This has a downside. Film is an art form; it is about imagination and creativity. Efficiency is important when the budget hits $700 million, but no one will pay to see just an efficiently made film. So the real challenge in using the new technology is to prevent the requirements of the computers from destroying the imagination of the artists.
Nevertheless, used well, the new technology promises a treat for movie goers. Films were always an attempt to blur the boundary between reality and imagination. With the new technology, imagination is winning.
John Fairall is editor of M& M
