Machineries Of Joy

  THE MACHINERIES OF JOY

  © 1984 by Greg Bear. All rights reserved.

  Introduction:

  In October of 1983, I traveled from San Diego to Los Angeles and San Francisco, researching a proposed article for OMNI Magazine. What I saw astonished me.... and influenced me heavily when I went on to write the novel-length Blood Music and Eon. Here was not the beginning of the computer graphics revolution, which had occurred decades earlier, but the beginning of the flowering of that revolution. I could hardly restrain my enthusiasm. I suspect the last few pages of this piece will date badly as time goes by, but they show my frame of mind. And the frames of mind of dozens of other authors, as well; the information age has taken science fiction by storm.

  OMNI never used this piece, although they paid me for it. Nor did they use the hundreds of pictures I gathered, a selection from which would have accompanied it. Many people gave generously of their time, yet never saw their names or ideas in print. I hope this publication pays them back in some small measure.

  The circumstances described below have, of course, changed considerably. Digital Productions has changed hands and management; Robert Abel and Associates is no longer an independent company. The revolution has become even more stimulating and promising. Its effects are everywhere.

  This article was completed in early 1984.

  THE MACHINERIES OF JOY

  "Dinosaurs!" The artist spreads his arms as if to embrace them. "I need the exact specifications--gridwork layouts of bones, muscles, scale patterns." The artist's office is covered with drawings of spaceships and alien beings, strange landscapes and mechanical diagrams. "If I have those, I can put them into the computer. We can program each muscle, make the skin ripple over the muscles. Tell the computer how they took a step, how they fought..."

  And once again, dinosaurs will walk and fight. The artist is living a childhood daydream: he has the power to bring dead creatures to life. Even more remarkable, he has the power-- with the aid of dozens of technicians, programmers and fellow artists--to film objects that have never existed in any material form and make them interact with live actors.

  But dinosaurs are a future project. The matter immediately at hand is a space battle. At night, within a stark white-walled enclave, the artist, director and technician sit before a video monitor, examining the progressive stages of a nonexistent spaceship's destruction. Highly detailed ships-- complete with crew--are dueling to the finish. One spaceship is destined not to survive; its hull is disassembled in the first of six boxes on the monitor. The early stages of an expanding blast are overlaid in subsequent boxes.

  The artist describes an explosion in space. "I'd like the whole screen to flash white for one frame. Next we see an opaque fireball--fuzzy at the edges--surrounding the debris." He demonstrates an expanding sphere with hand gestures. "Then we ramp it down to transparency as the fireball grows." (To "ramp" is to smoothly increase or decrease any function.) "When the shockwave passes, all the little stuff--gases and tiny fragments- -fly past and then we see the big scraps, a little slower, not as much energy." His grin is gleeful now. The director nods in agreement; this is, indeed, an explosion in space, not your usual smoke-and-fireworks exhibit.

  The stages of the explosion are being fed into powerful computers, isolated beyond glass walls at the opposite end of the studio in a pristine white-floored environment. Artist, director and technician are playing god games in an unreal universe.

  Ultimately, it is all numbers, points charted in a space of three dimensions within a computer. Each number represents part of the position of a pixel, or picture element, millions of which go together to form a shape. It is the computer's duty to keep track of the numbers, and the shapes they represent. Perspective, color, shadow, motion, must all be processed with scrupulous accuracy or the apparent reality will collapse.

  The numbers are then converted to signals which can be displayed on a monitor. The pixels assemble, and a spaceship is destroyed, frame by frame. When the result is printed onto film, it will be indistinguishable from very high-grade special effects accomplished with painstaking model work.

  It will look as real as anything else in the finished motion picture. The artist, director and technician are, of course, fictitious, and the scenario is a technological fantasy, not to be realized for years, perhaps decades to come--

  And if you believe that, you haven't been keeping track of recent advances in the incredible field of computer graphics.

  It is happening now.

  The artist is veteran production designer Ron Cobb, (ALIEN, CONAN THE BARBARIAN); the director is Nick Castle (TAG, SKATETOWN U.S.A.) and the motion picture is THE LAST STARFIGHTER, a joint Universal-Lorimar production. Under the auspices of Los Angeles-based Digital Productions, headed by John Whitney Jr., all of the special effects for THE LAST STARFIGHTER are being done by digital scene simulation--computer graphics designed to match reality. Using two powerful Cray super-computers and a phalanx of other machines, Digital Productions is taking a gamble--some say a big gamble--by committing itself wholeheartedly to the future.

  The future of computer graphics will be extraordinary. Most of the experts in the field--the best can still be numbered on two hands--agree that we are on the verge of a revolution perhaps more basic and disruptive than Gutenberg's movable type. Communications and education will be fundamentally reshaped. The entertainment industry will experience changes far more drastic than the transition from silent movies to talkies, and talkies to TV.

  The power that presently resides in the hands of a knowledgable few, will soon be available to all.

  But first, back to the numbers.

  The world of the computer is a very simple one. Everything is broken down into bits, a bit being the information required to answer any question with yes or no; in binary, yes equals 1, and no equals 0. Binary numbers consist of chains of ones and zeros. (In binary, 01 equals one, but 10 equals two.) More elaborate codes have been created to relate letters and symbols to certain numbers--thus allowing computers to display both numbers and text. Other codes relate the positions of glowing dots on a video screen using coordinates much like those on a map. A picture can be "digitized"--broken down into these numbered positions--and put into a computer, which can then manipulate the picture in a wide variety of ways.

  A picture can also be formed within the computer by charting key elements on a graph, feeding the computer coordinates and instructing it to draw lines or curves between the points. Mathematical equations which determine fixed geometric figures or curves can simplify the process; the computer can be instructed to draw a circle of a certain diameter around a point, or an ellipse; to trace out a square and expand it into a cube, and so on.

  In fact, a "space" is determined within the computer, having three or more dimensions, and any object can be described within that space, given sufficiently detailed coordinates. If the object is simple, like a cone, a "lathe" program can rotate a triangle around an axis to form a cone, or a circle can be turned around any diameter to create a sphere, much as a shape is spun from a block of wood on a lathe. More complex, irregular shapes take more complicated instructions, and much more time. Once the object is constructed in a simple line drawing, or "wireframe," additional programs can add a light source to give it highlight and cast a shadow. Colors and textures can be "mapped" on its surface. A point of view can be established, and what is not seen from that point of view--the back of the object--can be clipped, making it appear opaque and solid.

  The process seems simple enough, but in reality the work involved in creating real-seeming objects on today's machines is extensive. The most complicated methods of creating objects in a computer--such as a technique called "ray tracing"-- can take weeks of computer t
ime. Simpler techniques can reduce the time to fractions of a second, but with a corresponding loss of color, shadow and detail.

  Once the object's numbers have been fed into the computer, the computer knows what the object looks like from all sides, at any distance, in relation to any other object or perspective within the machine's memory. A nonexistent spaceship can be made to zoom past a simulated planet, approach a much larger "mother ship" and dock inside a highly detailed landing bay, all in perfect perspective.

  The computer can then display the objects in two dimensions on a video screen, or send signals to a printer to transfer images to film. Since the object has actually been mapped in more than two dimensions, the computer can be instructed to project two points of view, creating a parallax similar to that between our two eyes. The slightly separated images can be combined stereoscopically for a realistic feeling of depth.

  If the film image needs to be "squeezed" anamorphically onto 35mm stock for later projection on a wide screen, the computer can do that, as well. Any required lens can be simulated within the machine. In the 1950s, artists and programmers began to pioneer the techniques still being elaborated upon today. John Whitney Sr. was among the earliest, starting in the late 1940s. He later received the first IBM grant to study computer graphics in detail, and was installed in a ground-floor corner window of the IBM building in New York, displaying images for passers-by.

  Bill Fetter began exploring the possibilities of wireframe animation at Boeing in the late l950s, and assembled the first computer generated commercial in the late 1960s.

  In the early seventies, Ken Knowlton and Michael Noll came on the scene--Knowlton working for Bell Labs, and Noll arranging for the first gallery showing of computer art. Noll's specialty was simulating "clay paintings"--made with plasticine-- using computer images. Many viewers couldn't tell which were pictures of real clay paintings, and which were simulated.

  In the last ten years, the progress has been astonishing; around the world, computers are helping to create images for scientific research, education, fine art and entertainment.

  Sometimes the divisions between these categories are erased; the enchanting beauty of a moving computer image can turn a prosaic enterprise--such as stress analysis of pipe joins--into art. The most extensive use of computer animation has been in advertising. Already familiar to TV viewers are the plethora of "neon"-look commercials for banks, airlines and automobile manufacturers. Generically, computer animation relying on line graphics is known as "vector" animation. Using various animation techniques--inside and outside the computer--the lines of these "wireframe" drawings can be made to glow like neon tubes. This look has become so widespread that within the industry it is becoming a cliche, to be avoided if possible. Filling in a wireframe object with color, shadow and texture is called "raster graphics" or "raster" animation. This requires a more powerful computer, such as the Evans and Sutherland, or the Digital Equipment Corporation VAX machines commonly found in commercial studios. Some interesting effects can be obtained by fudging (not a technical term). The surface of an object to be vector- animated can be covered with "cross-thatching," using more lines instead of full raster graphics. This is known as "psuedo-raster" animation and can be charming, even though it falls in a middle range likely to be used less often as equipment and programming improve.

  Crude raster graphics can be judged by "aliasing"--the appearance of the "jaggies" along an object's edges. Each pixel stands out against a contrasting color, and when the object moves, the pixels can appear to march along the edge. These can be eliminated by coloring alternating edge pixels in shades that mediate between the contrasting colors. The border is softened slightly, and the graphics are said to be "anti-aliased."

  The most powerful computers available to animators-- the Cray series (the Cray 1, an expanded version called the Cray XMP, and a much smaller, even faster Cray 2) usually reside in defense establishments and major research laboratories. Digital Productions is the only private effects studio that owns Crays. The Cray corporation is reluctant to release the locations of all its machines, but it is well known that the Sandia Labs and Lawrence Livermore National Laboratory have a number on hand.

  By time-sharing--having the computers process their work when not otherwise busy--researchers in several such establishments have done important work programming computers to "understand" and draw transparent objects, lenses and realistic landscapes.

  Two of the most prolific of these researchers are James F. Blinn at the Jet Propulsion Laboratory in Pasadena, and Nelson Max at Lawrence Livermore National Laboratories. Blinn's group at JPL animated the striking computer simulations of the Voyager probes' journeys to the outer planets, widely shown on network and public television in 1980-81. Nelson Max has worked largely on graphic representation of biological processes. Using his graphics programs, he has been able to predict how molecules will interact before lab tests have been made. Max has also investigated the effects of mutagens on DNA, and modeled the structure of very tiny viruses.

  After months or years of painstaking labor, computer artists display their wares at annual SIGGRAPH conventions. (SIGGRAPH stands for Special Interest Group, Graphics, a division of the Association of Computing Machinery, or ACM.) Private individuals, employees of giant research establishments and commercial film studios gather to compare notes and keep up on the latest developments.

  C.P. Snow's "Two Cultures" are inevitably wedded in computer graphics.

  Not since Leonardo da Vinci have so many technical disciplines been required of working artists. Not only must they have basic drawing and drafting skills, but they must know at least the rudiments of programming. They must understand how light reflects, refracts and diffuses--and be able to translate their knowledge into terms the computer can digest. The artist can no longer stand aloof from science and math. New techniques can take him to the frontiers of theory. Recent work in the texturing of surfaces has used fractals, mathematical entities capable of generating very complex patterns. Perhaps the most familiar example of computer animation with fractal-generated landscapes is the "Genesis" sequence from STAR TREK II: THE WRATH OF KHAN, made for Paramount Pictures by Sprockets, the computer division of Lucasfilm's Industrial Light and Magic.

  One of the focal points for computer animators was the Walt Disney production of TRON. Information International, Inc., (known as triple-I), Mathematical Applications Group, Inc. (MAGI) Robert Abel and Associates and Digital Effects all contributed their expertise; yet TRON contained only ten to fifteen minutes of full computer animation. The rest was accomplished with conventional special effects and animation techniques.

  A great many of the people who worked on TRON have now moved on to positions in companies around the country. A few, such as Richard Taylor, are still involved with feature-length motion pictures. Taylor is reportedly hard at work on a film called DREAMER for Paramount. In advertising, two of the biggest film companies have made a major commitment to computer graphics. Robert Abel in Hollywood--long renowned for the beautiful combinations of live action and back-lit animation in his Levi's and Seven-Up commercials--assembled a computer graphics division while assigned to do special effects for STAR TREK: THE MOTION PICTURE. Unlike Digital Productions, however, Abel kept all his other special effects techniques, considering computer graphics as another tool, not an end in itself. "A lot of the stuff we do is combination," Abel explains, "where we combine miniatures and live action with computer images." Pure computer animation, at present, is more expensive than many other techniques, and in Abel's view, flexibility and variety are necessary to the production of commercial advertising films.

  Bo Gehring, in charge of Bo Gehring Associates in Venice, California, originally came to the west coast to do computer animation tests for Steven Spielberg's CLOSE ENCOUNTERS OF THE THIRD KIND. The tests proved unsatisfactory but Gehring stayed on to found his own company--again, with a complete spectrum of techniques at his disposal. Unlike Abel, who began as a documentary
film maker, Gehring's roots are in computer graphics, but he agrees with Abel that commitment to one technique is risky. As for getting involved in feature films: "Ninety million dollars is spent each day on advertising in the United States," Gehring says. "Feature films can't begin to match that level of financing. I'm secure where I am."

  Both Gehring and Abel believe that computer graphics is still in its infancy, and will probably have a major effect on all forms of visual communication. For the moment, however, neither is willing to make the leap of faith required for an operation such as that being conducted at Digital Productions. And truthfully, Gehring admits that his financial backing is not equal to Digital Productions', which is supported by Ramtek, a major computer company. "I am a bit envious of what John Whitney Jr. and Gary Demos have come into at Digital--all that [computing] power. But I'm happy with my situation, and just can't see taking that kind of risk right now."

  Gehring also expresses an interest in digital sound synthesis. "I'm one of those people who has to pull off the road when something really intriguing comes on the car radio. I firmly believe that sound is at least the equal of sight in bandwidth-- complexity of information--and synthetic sound is a fascinating area that's barely been explored." Another of the Big Three companies, R. Greenberg in New York, is rapidly building its computer graphics division.

  Computers have revolutionized the film industry in many more ways than computer graphics. Virtually all commercial studios, whether producing advertising or feature films, use computers to control complex camera movements or integrate different elements of photography. At Robert Abel, slit-scan photography is a staple item. The process was originally developed by Con Pedersen and Douglas Trumbull while working for Stanley Kubrick on 2001: A SPACE ODYSSEY. Pedersen now works at Abel, where he supervises other aspects of special effects production, including computer graphics. (Trumbull, interestingly, seems to eschew full computer animation. In his recent film BRAINSTORM, even sequences which appeared to be computer-generated were done using other techniques.)