Last month I complained that the Hyperloop-alpha proposal has a heat balance problem. Here is the fix for that problem, as promised. I’ve broken up my changes to the design in a series of steps, to make it easier to see that each step is individually beneficial.
Step 1: Hyperloop + 3 MPa steam heat dump
My numbers don’t exactly match Mr. Musk’s. It seems we have slightly different specific heats for low pressure air, among other things. This diagram should serve as a base case which can be compared directly to the diagram from the Hyperloop-alpha proposal (check the last blog post).
I’ve detailed the steam heat dump system to run up to 3 MPa. Water is sourced from a small tank and steam is dumped into a much larger tank. The tanks are connected, so that as the steam pressure rises so too does the water pressure, which should reduce pumping losses. Eventually even the small tank is completely filled with steam. I've accounted for the two tanks as a single tank here, as the wall-in-the-middle detail should not affect the mass budget at all.
As I predicted in my last post, the heat dump tank is far too big... at least 12 meters long. The tube taking bypass air to the rear nozzle is also far too big, and will interfere with the passenger compartment. Finally, the battery to run the turbines is gigantic.
Step 2: ... + bypass intercooler
By picking off the air output from before rather than after the first intercooler, the heat load is radically reduced to practical dimensions. The heat going through the first intercooler is not a typo... it's really been reduced to 10% of the prior value. Part of this reduction comes from balancing the pressure ratios across the two compressors. Unfortunately, the bypass tube is now even larger and less practical.
Step 3: ... + Reduced Tube Pressure
Reducing the pressure in the transport tube to 25 Pascals decreases the compressor power, but oddly increases the amount of cooling required. That's because we're still cooling the same mass flow of air, but compressing that air more.
The cooling tank gets larger, but note that the battery gets cut in half, for significant savings in mass and cost. The bypass tube is smaller but still too large.
Reducing tube pressure should not cause problems with the vacuum pumps. 25 Pa is considered a "rough" vacuum, and corresponds to an altitude of 59 km, where helium balloons operate. The mean free path in this gas is around 0.3 mm, which is small enough relative to vacuum pump turbine blades that the gas still acts like a gas. Ordinary vacuum pumps will work fine.
Turbomolecular pumps, which use different aerodynamics than regular vacuum pumps, generally start operating at 10 Pa or less. I am not suggesting we drop into that range.
The intercoolers above are removing nearly all the energy added by the turbines. We can avoid adding so much energy in the first place if the turbines operate on cryogenically cold air. This is a big step, but it's necessary because the capsule design in Step 3 is too bulky.
To do it, we swap the water/steam heat dump for a more compact but substantially heavier liquid air heat dump. Boiling liquid air does not absorb as much heat as boiling water, but it also does not expand as much. For the same heat absorbed, the tank weighs less but the fluid weighs far more.
The real magic, however, is that the heat is absorbed at a much lower temperature. The intercooler output will now approach the cold sink temperature of 75 Kelvin. Once again, the battery loses more weight than we add to the heat dump, for a net savings of cost and mass.
The high speed, low density intake intercooler here is similar to but much less extreme than the one proposed for the Skylon spaceplane. It is not completely settled technology but much of the R&D has already been done. In particular, one of the issues would be clogging from frozen water and CO2 ice. In an operating system, the tube air would be dried by prior passages of capsules. So clogging would mainly be an issue when restarting.
The heat dump can also be used as a source of air. For instance, at low speeds the bow intake compressor may not supply enough air for the air bearings. Make-up air can be taken from the heat dump. In an emergency, while waiting for the tube to repressurize, air from the heat dump tank could be used to supply 20 occupants with breathing air for 20 hours.
I think this design largely achieves what Mr. Musk was attempting to accomplish.
The Hyperloop proposal implies a steam tank almost as large as the entire vehicle, and at least 3200 kg, which is not accounted for in the Hyperloop mass budget. This is a design breaker.
The capsule as configured is using most of it's power to drive a refrigerator which cools the air in the tube. This refrigerator is not mentioned as such.
The refrigerator, as configured, will not cool the tube air back to it's ambient temperature. But refrigeration is unnecessary.
I understand that Elon commutes from Los Angeles to San Francisco every week, and that the high speed rail project will not help him. I imagine that his frustration with this commute is exactly what has kept bringing his attention back to this Hyperloop thing for so long. I'm sure the boards at both SpaceX and Tesla have threatened to fire him if he starts another high-risk startup. The guy has a tough life. :)
The Hyperloop proposal has one big idea that I like: the air bearings, rather than the usual magnetic levitation. Air bearings are a well developed technology, and require much less capital in the track. I had not realized that air bearings were a feasible idea for high speed transport, as this is the first I've read about their air requirements at high speed. (I called three air bearing companies to verify Elon's numbers, and none of them could offer any guidance on air consumption or even stable operation at near-sonic velocity.) Elon is implying a 2000:1 lift/drag ratio for the passenger version, and a 2500:1 lift/drag ratio for the vehicle version. That's incredibly good. Maglev lift/drag ratios are generally rise to 200:1, which is considered really good. The Livermore maglev requires substantial forward speed before it gets that efficient. If Elon's numbers are real then air bearings are awesome... but I'd like to see some evidence.
I don't quite understand Elon's bearing drag power numbers. Drag power is usually just vehicle speed times the drag force. He's got an extra factor in there which adds about 10%, and I'm not sure where that comes from. It's not compressor power, because that is much greater. If we add the compressor power to the drag power, we get L/D ratios of 426:1 and 634:1 for the two configurations. Those are still game changing numbers. These numbers matter because if the air demand of the bearings is higher, the compressor power will increase. If the compressor power increases too much, the battery will get too big and the Hyperloop idea will not work.
The Hyperloop proposal has another big idea that I don't understand: the Kantrowitz limit. It's clearly a big problem, because the drag coefficient implied by Elon's numbers here is 3.95. (That number is suspiciously close to an integer, and suggests that the drag numbers are back-of-envelope and not the result of simulation.) Compare that to a Tesla S (0.24), or a bullet (0.29). The only saving grace is that his predicted aerodynamic drag (910 N for the vehicle-carrying capsule) isn't too much more than the bearing drag (187 N), even if you include the prorated compressor power in the latter number (740 N).
At one point, Elon insists that propulsive power should be delivered through the track, rather than from the vehicle. But his propulsive power (628 kW) is less than the compressor power (868 kW) that the vehicle is already signed up to provide. It might be simpler to just let the vehicle do the whole job and let the track be passive. He's already suggested using huge battery packs to deliver the propulsion power, so the only change is to put those packs on the vehicle. This is a nit, let me get on to my main point.
The proposed bypass refrigerator scheme is fascinating but flawed. Here's the flow diagram for the vehicle-carrying Hyperloop. There is a typo: the air coming out of the second compressor should be at 594 K, not 59 K, assuming that second compressor is 72.4% efficient like the first one is.
There are much bigger problems than typos here.
The air in the tube is not going to be 19 C. Those Hyperloop capsules dissipate 868 kW of compressor power and 285 kW of propulsive power. As drawn above, that propulsive power is going to be primarily dissipated into the air in the tube, heating it tremendously. There is also 59 kW of bearing drag that will be dissipated into the tube wall and somewhat into the air.
As drawn above, the intercooler is actually acting as a refrigerator. The air expanded out the back would leave at around 126 Kelvin... 147 degrees C below freezing! This will suck 209 kW of heat out of the tube air (49 kW for the passenger-only version).
The intercooler refrigerator does not balance the propulsive drag heating. Combining the two, I find that the vehicle will still dump 76 kW into the air, heating the air in the tube by 24 C (35 C for the passenger-only version). By increasing the intermediate pressure and intercooler heat load a bit, this problem can be fixed.
There is no way the output temp of a lightweight intercooler will be within 7 C of the incoming water temperature.
The water is absorbing 852 kW, or 2185 J/g. The water flow rate should be 327 g/sec, not 390 g/sec, to fully boil the water at normal atmospheric pressure. But that's wrong too...
To make the steam tank reasonably small, the steam will have to be held at high pressure. Boiling 20 C input water at 450 psi would absorb 2800 J/g and produce steam at 507 K. The water rate would then be 304 g/s, the tank would be 638 kg and 42.5 m3. That's clearly far too large (it's as big as the whole vehicle), but that's the largest pressure my steam table goes to. Even if the steam is heated to 857 K (supercritical), I don't think the steam tank will be small enough to work.
The steam tank will be heavy. Assuming carbon composite overwrap construction and 100 MPa working strain, the steam tank will weigh 3200 kg. Changing the steam temperature and pressure conditions will vary this a little, but not much.
As a first step, I would take the bypass air directly from the axial compressor, which will remove about 80% of the heat load on the intercooler. A 5x reduction in the size of the steam tank will get the mass budget back under control, but the steam tank will still be awkwardly large. As a bonus this should deliver 70% more thrust, but that's still too small to matter much.
It should be possible to dump a fair bit of compressor heat from a radiator on the surface of the capsule. I think this can reduce the heat load on the intercooler and steam tank by another factor of two, which should bring the steam tank volume into a more reasonable range. It will still be a major component in the vehicle design.
As a next step, the vehicle can be slowed, perhaps by 10%, which allows some of the air in front of the vehicle to actually accelerate up to the alpha proposal mach .91 number as it goes around. The compressor inlet and mass flow can then be downsized since it need only handle a portion of the airflow. I think it should be possible to cut the compressor power numbers in half this way.
In my next post I'll work the numbers on these suggestions.
This isn't a movie review. There are already lots of movie reviews for Pacific Rim.
Rather, I want to ask the question, why was this movie weak in the ways that it was?
First: suspension of disbelief. If faced with 200 foot tall monsters that kill tens of thousands and wreck billions of dollars of equipment and property, we all know the US military will respond. First with battleships
and torpedoes.
If that doesn't work, we'll try this, especially since the target is in a known spot well away from large population concentrations.
Those are the hammers we have. The film did not suggest any reasons why these things were tried and failed. Since the target audience is quite familiar with these things, that's a problem.
I personally needed to see something about no explosions or shock waves near a Kaiju. Something that knocks out all the standard responses. This would get rid of range weapons, and turn the Kaiju/Jaeger fight into a brawl with knives, teeth, and claws, which is what we want.
The movie has a germ of a good idea: no electronics near Kaijus. This could have been seriously cool, because it means you must use a person as the control system. I think the idea of multiple people would have been even better with NO electronics and NO drift. Advanced puppets are driven by multiple people. It takes teamwork, practice, and real, visible communication, which would have worked better for communicating a storyline.
The film has another good idea: the Kaiju keep getting bigger over many years. This is fabulous, because it gives us time to develop Jaegers.
For me, the point of the film was to develop a visual language for BIG. It worked where the film used familiar references. Cars, streetlights, trucks. Water coming off the monsters worked well because water has a characteristic scale.
Once the scene moved underwater, we lost all of these familiar references. The Jaegers and Kaiju looked small underwater.
Lifting the Jaegers with helicopters, and having them inside large strong enclosures, violated the language. To be lifted by helicopters, they would have had to be fluffy. I don't want fluffy Jaegers. I want Jaegers that sink in water: denser and more heavily armored than a battleship.
And the enclosures for the Jaegers were wrong. The point of building something really big is that there isn't something bigger. Really big stuff, like a battleship, is built outside. Things like battleships don't need to be protected from the weather, and the folks that work on them don't either. Perhaps they use portable shelters. You have gantry cranes for lifting pieces into position, like this:
Finally, go take a look at that battleship pic at the top of the post again. Do you see any big air intakes? No. That's because air intakes and turbines are flimsy things that have to be carefully engineered against bird strikes. The big turbine intake on Gypsy Danger looked flimsy too. That's a problem.
I think the Jaegers should have been actuated by steam cylinders, so that as they moved great billowing clouds of steam shot out of the joints. That steam would be heated by a nuclear reactor. So we'd get explosive movement and a visual style like this:
Google just announced project Loon (Google Blog, AFP article, and Wired article). This is a scheme to provide internet access to folks in the southern hemisphere from high altitude balloons. Short take: they can steer the balloon, which is potentially the required gamechanger, the balloon is too big for their stated payload, and it'll need to get bigger if they want to build a global WiFi hotspot.
This is Google's second try at the southern hemisphere problem. In 2008 they helped start O3b Networks, which is attempting to launch a low earth orbit satellite constellation that would provide broadband access.
I'll start with a small point. Something is odd about the size of the balloon. The stated payload is 10 kg and the stated balloon is 15 meters in diameter, 3 mil thick polyethylene, and operates at 60,000 feet. That's too big a balloon for that payload.
The stated balloon envelope will weigh about 20 kg. So, they'd need about 30 kg of lift, which is about 30 m3 of helium at standard temperature and pressure. But at 60,000 feet, the ambient density is around 100 g/m3 and helium density is around 14 g/m3. So that's around 360 m3 of volume. Google's 15 meter balloon holds 1767 m3 and is sized to pick up something a lot bigger.
So, I'm guessing they actually want to lift something more like 100 to 150 kg. The helium to fill that is around $1300, the balloon is another $1500 in quantity, and the payload will be around $2k (but see below).
I wondered if they'd lose helium fast. It turns out, no. According to this table, the helium permeability of polyethylene is 5.3e-8 cm^2/sec. The 'Loon probably has 235 m^2 of 75-micron-thick polyethylene, and so leaks 1.4 m^3/day. I'm surprised at how small that is... sounds like they could stay up for a year if the balloon is fully inflated and residence was limited by leakage. My guess is that time aloft will be limited by ultraviolet breakdown of the polyethylene and by wind shear occasionally ripping the things apart.
Assuming that ground terminals require a line of sight to the 'Loon at least 10 degrees above the horizon, each 'Loon covers a circle 200 km in diameter. Uniformly covering the whole earth would take 16,300 balloons. But they aren't going to cover the whole earth, and they had better not cover it uniformly.
The first big rollout, if it ever happens, will be over the Hadley cell between 30 and 60 degrees south of the equator, and will cover primarily Australia, New Zealand, Argentina, South Africa, Chile, and Uruguay. They'll need around 3000 balloons to get complete coverage. The second big rollout will be over the Hadley cell between the equator and 30 degrees south, will require 4000 balloons, and will primarily cover Indonesia and Brazil. There will be major objections to these overflights in all the affected countries, and I'm not sure how Google will try to overcome those. But covering 1 billion people for $15m is pretty awesome.
The next logical rollout would be over the Hadley cell between the equator and 30 N. Once again, it would take 4000 balloons and cover India, the Philippines, most of central Africa, everything from Columbia to central Mexico. I just can't imagine this ever being accepted by governments with the ability to shoot down unwanted overflying balloons. The next Hadley cell north gets China and the US and is even less politically feasible.
The 'Loon has a proprietary link to the ground stations. Why not WiFi? The problem is that a single 'Loon will often cover 100,000 cellphones, each with a WiFi endpoint. Without some way to separate all those signals, the 'Loon will just see noise. A proprietary link allows Google to throttle the number of simultaneously transmitting terminals, and add coding gain, to get enough signal to noise ratio to communicate over dozens of km. Basically, the proprietary link is a way to shake the tree and see how governments react to a potentially uncensorable global ISP.
The better solution, the one I'm sure the Google engineers would love to implement, would be to put 802.11ac on the 'Loon. 802.11ac is the next-generation wireless Ethernet standard, and it will be on all smartphones in two years. Crucially, the standard protocol requires the handsets give the access point the feedback necessary for the access point to use phased array antennas to form beams. Those beams have two big consequences.
The beams let the 'Loon capture more of the energy transmitted from the handset. A 6 m diameter phased array should capture enough energy from a cellphone at 40 km to enable 2-3 Mb/s transmission. I don't know that 802.11ac has a LDPC code that lets it go that slowly, but if not, Google may be able to require that Android handsets implement an additional optional royalty-free code.
The beams let the 'Loon distinguish between tens of thousands of handsets simultaneously. A 6 meter diameter phased array could implement cells on the ground 250 m diameter (right under the balloon) to 500 x 1000 m (far away from the balloon). This density isn't going to let folks in Cape Town watch HDTV via YouTube, but it'll handle email and web surfing just fine.
It's interesting to think about the parallels between this idea and the 1990s idea of having a constellation of low earth orbit satellites providing broadband access. At least five different schemes were proposed. Iridium and Globalstar, both providing cellphone coverage, were actually built, and both went broke. Teledesic, backed by Bill Gates, Paul Allen, and Saudi prince Alwaleed bin Talal, got canned before it launched it's constellation. The problems with all these schemes were:
The handsets were bulky and heavy because the satellites were 400 km up and 1000 km away. 'Loon cuts the range by more than an order of magnitude, which cuts the antenna size on both the access point (balloon) and handset by an order of magnitude.
The constellations required custom handsets. 'Loon has this problem too but 802.11ac is a path to a solution.
The access point hardware went on satellites which are expensive and hard to maintain. The balloons should be easier to maintain... than satellites. Though it's better than satellite constellations, I'd still count this issue against 'Loon.
The orbital geometry meant that coverage was concentrated near the poles, where there aren't many people. 'Loon does not have this problem.
The schemes required access point hardware sufficient to cover the entire earth, but only provided value where there were customers. 'Loon has this problem even worse than Teledesic. The Earth as a whole is about 30% land, but the southern hemisphere is 19% land. Worse yet, only 12% of the world's population lives in the southern hemisphere. This problem can kill 'Loon.
There are currently two broadband LEO constellations in the works. O3b, as mentioned above, and COMMstellation. I don't see how either has fixed the problems that sunk Teledesic et al.
There has also been a bunch of work on broadband from High Altitude Platforms. Balloons have been considered before, as havehigh altitude, long enduranceaircraft, and interestinghybrids between the two. In the late 90s, Angel Technologies was going to fly a Rutan-designed aircraft in the stratosphere, carrying a phased array broadband access point. Aircraft can carry larger payloads (Halo could carry 1 ton), provide more power (Halo could provide 20 kilowatts), and can keep station over populated areas. Stationkeeping fixes problem #5 above, and is a huge deal.
'Loon may have a new answer to problem #5. They can steer. Fast forward to 1:11 in this video.
Steering is a big difference from satellites. Stratospheric winds will carry the 'Loons eastward around the globe, but if they can steer north-south while that is happening, they may be able to crowd the 'Loons over denser populations, and scoot across the oceans on the jet stream. Steering could dramatically reduce the number of 'Loons needed.
Technology developments to watch for:
A solar power array that faces nearly sideways, steered toward the sun with fans or something. Google is going to need this for the first big rollout, since the sun will not be more than 20 degrees above the horizon for most of the winter, and the grazing angle will kill their panel efficiency.
An ASIC that enables a 802.11ac access point to handle 1000 transceivers in less than 5 kilowatts. They are going to need this chip for the last forseeable rollout, the one that interfaces directly to cellphones and turns Google into a global ISP for Brazil and Indonesia at least. The ASIC is required because this generation 'Loon will be constrained by power. Power requires big solar arrays and batteries to get through the night, and those require a bigger, stronger balloon.
A balloon-to-balloon optical link consisting of multiple small gimbal-stabilized telescopes, perhaps two inches in diameter, to relay the photons between 10 gigabit ethernet fiber optic transceivers. I've wanted to see this technology for years, but the killer app hasn't shown up yet, because at ground level the weather would frequently disrupt any link. At 60k feet that shouldn't be a problem. They'll need this technology for that last big rollout. I don't think radios are going to work well enough.
As a camera nerd, I can't help but note the temptation to put a 2 kg camera on each 'Loon. You'd get something like hourly coverage of everywhere at 12 inch resolution, and real-time video coverage of smaller targets (like traffic accidents) as well. There is a real market for that... SkyBox Imaging just got $90m in venture capital to address it. This imagery, piped into Google Maps/Earth, would provide the live view of the entire earth that everyone already expects to be there.
[Update: I'd like to thank Phil Pressel, John Gregor, and Gordon Petrie for their corrections to this post. The changes required have been so extensive I have not marked them.]
[Update 2: Phil Pressel just released his book Meeting the Challenge: The Hexagon KH-9 Reconnaissance Satellite (get it at AIAA or Amazon). I've ordered it and will post a review after I get it.]
From 1957 to 1965, a high tech startup called Itek made the world's most sophisticated satellite reconnaissance cameras for a single customer -- the CIA. The company has a fascinating subsequent history, as they ended up building cameras for Apollo and Viking. Eventually they ended up building the DB-110 reconnaissance pod, which I'll do a blog post on some day.
Merton Davies at RAND apparently originated the idea of using a spinning camera mounted on a spin-stabilized satellite to take panoramic shots with a narrow-angle lens. Amrom Katz passed the concept to Walter Levinson at Boston University Physical Research Laboratory (BUPRL). Levinson refined the idea to that of using an oscillating lens, for use in the HYAC-1 panoramic camera in Air Force high altitude reconnaissance balloons. Itek, just a few weeks after incorporation in late 1957, bought BUPRL, which was developing the HYAC-1 panoramic camera to take pictures from high altitude balloons.
Soon after, the CIA contacted Itek to discuss the camera requirements for the first spy satellites. All of these initial satellites came to use the rotating panoramic camera. I think this is the KH-4A or KH-4B Corona.
Itek also built versions of these cameras for use in the U-2 and SR-71 aircraft, in which they were called the OBC (Optical Bar Camera, named for the appearance of the field of regard on the ground). These were first used in the 1960s and are still in use today. Here is an Itek Optical Bar Camera that goes in the nose of an U-2:
Take a look at the big, wide blue bar under the airplane. That's what a single frame of the camera shot. It's huge. I've heard that this "bar" looking frame was why they called it the optical bar camera. However, the NRO's Hexagon documentation refers to the rotating optical assembly (what in many cameras is called the "Optical Tube Assembly") as the optical bar.
After a string of successful programs, requirements racheted up and tensions grew between NRO and CIA over the next program, Fulcrum. Early on, Itek was contracting with both the NRO and the CIA on competing projects. Itek pulled out of the CIA's project, and some combination of the NRO and the CIA took all their (government-owned) work and gave the job to Perkin-Elmer. When the dust settled the project was renamed Hexagon. Perkin-Elmer built the KH-9 Optical Bar Camera with their own design rather than Itek's, as they didn't think the Itek design would work. Here is a look into the aperture of the KH-9 Optical Bar Camera.
The Itek OBCs in the U-2, SR-71, and Apollo spacecraft all had a rotating structure called the roller cage, which I suspect was fixed to the rotating camera. The Perkin-Elmer design in the KH-9 deleted the roller cage and the rotating fold mirror inside it, and instead had a servo controlled twisting platen.
Here is a comparison of various optical bar cameras built up to 1971 (the launch of the first KH-9).
Take a look at the area collected by the KH-9. The Soviet Union was a big place: 20m km2. Each of the four film re-entry capsules could return the entire USSR, in stereo, with plenty of side overlap and margin for images fouled by clouds. Typically they chose to take more frames with smaller swaths (90 degrees or 60 degrees) to get higher average resolution, which brought down the total take somewhat to around 60 million km2.
My resolution numbers up there are slightly inflated. The film used could only eke out 130 lp/mm when given the maximum possible amount of contrast, such as would be seen at a shadow line. For finding something like paint on a road they were about half that. Pixellated sensors have a much more drastic cliff, of course. So the e.g. KH-9 resolution above might be compared to anything like a 20 to 30 gigapixel camera today. I'll note that I don't have any of those to suggest.
There are two big concepts here that I think are important. The first is the mechanical and logistical difficulties of using film. Below I've spelled out some of the details. The second is that despite these headaches, until very recently, film has been superior in many ways to electronic sensors for massive survey work.
The basic problems with using film stem from the fact that the sensor surface is a thin, pliable, moving, relatively inexpensive object that has to be held within the camera with very high precision. There must have been several problems associated with getting the film aligned within +/- 11 microns of the lens's focal plane. Among other things, the machines resemble Van de Graf generators, and so the film is subject to static electricity buildup and discharges, along with heating that tends to make it even more pliable and sticky. To reduce the static buildup, many of these cameras slid the film over surfaces with hundreds of pores lifting the film with pressurized air.
I think the Itek designs spun the entire optical assembly at a constant rate. The entire spinning optical assembly is inside a gimbal which rocks back and forth 1.6 degrees in each cycle. The rocking motion accomplishes forward motion compensation, so that the sweep of the slit across the ground is orthogonal to the direction of flight. This compensation ensures that the motion of the image on the film moves almost exactly with the film, and there is no blurring in the direction of flight during longer exposures. This rocking motion must have required fairly large torques, and I'm sure this is one of the things that the Perkin-Elmer folks balked at when considering the Itek design in space. Note that a constantly rotating camera sweeps at the outer edges faster than at the center, so the forward motion compensation probably had to vary it's rate of rocking as it went to compensate.
Here is a diagram which shows how the film was wrapped around the roller cage in the Itek designs. As the optical assembly (including the roller cage) twists counterclockwise, the film is transported clockwise.
However, even with a constant spin rate, the film does not transport at a constant rate. For instance, in the SR-71 OBC, a frame is exposed for 640 ms, during which time the film rips around the roller cage at 2.9 meters per second (that's what the rollers see). For the next second, the film advances at just 1 meter per second, so that the frame boundary going across the top of the roller cage can meet up with the slit going around the bottom. Because of the speed change, many of the freewheeling rollers on the roller cage will contact unexposed film coming from the framing roller with a tangential speed difference of 1.9 meters per second. As each freewheeling roller changes speed to match the film, it seems to me it would tend to scuff the film. I'm sure they made sure to make those rollers as lightweight as possible to reduce their rotational momentum.
Note the unlabeled slit after the second mirror right next to the film wrapped around the roller cage. Only the portion of the film after this in the optical chain has light on it, so this is the only spot that must be held accurately. I don't really know how it was done, since every moving belt of material that I've even seen has vibrated. They may have had a glass reseau plate that the film slid across. Sliding film across glass at 2.9 meters per second seems like an excellent way to scratch one or both. I have no evidence for it yet, but this seems like a good application for the compressed air film handling scheme.
The Itek forward motion compensation gimbal also took out airplane pitch motions. Airplane roll motions were taken out by varying the optical tube roll rate. That's easy enough (it doesn't require torque), but the film rate supplied to the roller cage assembly in the back also had to change to match.
That last diagram gives a pretty good clue to another challenge in this design -- film curvature. Although I've not found any labelled dimensions, it looks like the roller cage in the Itek designs was about 300 mm in diameter. The design of the roller cage really had me puzzled for a while, because the film transport path is cylindrical, but the film being exposed by the slit has to be flat. I finally figured it out when I took a good look at this photo of the Apollo OBC (also made by Itek):
The roller cage is a cylindrical cage of ROLLERS! Duh! As the film passes between the rollers, it's locally flat, and that's how they keep the film surface matched to the lens focal plane. Here's a 5x blowup of the roller cage in the picture above. It looks like the rollers are packed together as close as they can get in order to minimize the polygonal variation in film distance from the center of roller cage rotation. I think this variation leads to a (small) variation in film speed at the exposure site.
There should be a spot in the roller cage where there aren't any rollers, and the film becomes planar for a while. This spot would be over the exposure slit. In the Apollo OBC pictured here, the gap between the rollers must be at least 7.6mm, and given the orientation of the lens at the top it should be on the back side of the roller cage that we don't see here.
The film is pulled taut around the stuttering, bucking and twisting roller cage with an assembly that looks like the following. During the exposure film is pulled around the roller cage at 5.7 meters/second. Here's the film path:
If the tension on the film is too small, small irregularities in it's "set" curvature will make it lift off as it goes around the roller cage, and small patches of the film will lift away from the cage. With a +/- 11 micron depth of focus, it doesn't take much lift off to make a problem. If the tension is too high, the film will wrinkle longitudinally.
The Perkin-Elmer design did not have the roller cage or the gimbal. Instead, they had a twisting platen assembly at the focal plane. This would twist back and forth through 130 degrees as the optical bar rotated through 360 degrees. The two were nearly locked together through the 120 degrees of rotation that were used for exposure.
Because the Perkin-Elmer design has no rocker gimbal doing forward motion compensation, and the optical assemblies rotate at constant speed, the sweep is faster at the edges than in the center, and the area swept in each frame is slightly S shaped. They may have splayed the roll axes of the optical bars to get first order forward motion compensation, but this doesn't change the S shape. To keep the image from smearing across the film, the KH-11 has to keep the slit perpendicular to the motion of the slit across the ground, accounting for the changing sweep rate versus spacecraft velocity, as well as the rotation of the Earth, which is 6% of the spacecraft velocity at the equator.
This is why the twisting platen in the KH-11 is servo locked to the twisting optical assembly. They have to vary the relative twist of the two a little bit to keep the projected slit perpendicular to it's projected motion.
After the picture was shot the film sat around for a month in a temperature and humidity controlled environment, and then was dropped, recovered, and developed. There was a lot of opportunity for the film to shrink differentially. All the mechanical twisting, as well as film shrinkage, must have made photogrammetry a nightmare.
The Sunny 16 rule says that you can properly expose ISO 100 film in bright daylight conditions with a f/16 lens with a 10 ms exposure. The KH-9 used mostly monochrome Kodak 1414, which has an Aerial Film Speed of 15, which I think is equivalent to ISO 38. In full sun a 1 ms exposure at f/3 would have worked fine. On the Apollo OBC that corresponds to a 1.9mm wide slit. They did have exposure control hardware, and it looks like they could exposure for two stops dimmer than full sun. They might have also stopped down from that, in order to avoid blowouts over ice.
At this point, I'm sure those of you who have worked with digital sensors are feeling pretty smug. But consider how wonderful film was for capturing and returning large amounts of imagery.
Each of the four re-entry vehicles on the KH-9 would bring back 5,000 36 gigapixel images. If somehow compressed to 1 bit per pixel, that would be about 20 terabytes. These days that's about 10 hard disks, and would take about three months to downlink at a constant 25 megabits per second. Since they were returning these re-entry vehicles every few months, a modern downlink is only barely adequate. It has only been in the last 10 years or so that disk drive capacities have become high enough to fit the data into the payload of one of those re-entry vehicles -- 30 years after the KH-9 was originally deployed.
In 1971, the area of film actively integrating photons in the KH-9 was 155 mm x 15 mm. The largest, fastest TDI CCD sensors commercially available in 2013 are 64 mm x 1.3 mm. The pixels on these are 5.2 microns rather than 3.7 as on film. The smaller integrating length (1.3 mm versus 22 mm) gives a maximum exposure time of 240 microseconds, which is smaller than the 2 milliseconds we would prefer. 155 mm x 10 mm CCDs with 3.7 micron pixels are not commercially available, but could probably be custom made.
Another issue would be the readout rate. A fast TDI in 2013 reads out lines at 90 KHz. The KH-9 was exposing film in the roller cage at 5.5 meters/second, which corresponds to a line readout rate of 1.5 MHz. This could be achieved with a custom built TDI sensor maybe 5 years ago. It would require 1300 ADCs running at 45 MHz, which would burn a lot of power. This might be possible with interesting packaging.
The capture rate of the KH-9 was so enormous it completely overwhelmed the ability of the photointerpreters at the CIA to examine the photographs. It's only recently that computers have gotten large enough to store and process imagery at this volume, and it's not at all clear that anyone has yet developed algorithms to find the "unknown unknowns" in a bunch of raw imagery. I think they call this "uncued image search".
To my knowledge the U.S. never again fielded a spysat with the ability to survey major portions of the earth at high resolution. Later Keyhole satellites appear to have concentrated more on taking valuable single shots at higher resolution (7 cm GSD), and on having the orbital maneuverability to get those shots. I think the intelligence folks lost interest in survey satellites when it became clear that they couldn't take advantage of the comprehensive coverage which was their primary feature. It's kind of ironic that the very problem that Itek was founded to solve (managing the huge volume of survey photography) ended up being a major reason why satellites with survey capacity made possible by Itek's cameras faded away. It's fascinating for me to see what has become of this problem.
Brian McClendon is on record as saying that Google has 5 million miles and 20 petabytes of Street View imagery. That's the processed imagery, not the raw take. The KH-9 raw take that overwhelmed the CIA photointerpreter capacity 30 years ago was less than 1% of what Google Street View shot. Unlike the KH-9 imagery, most of which I suspect has never been looked at, every one of the Street View panoramas has been seen by a real user. And Google is hardly the only organization using Big Data like this. The consumption problem that the CIA and NRO never solved has been utterly crushed by organizations with entirely different goals. (Granted, uncued search remains unsolved.)
Now that Big Data has caught up with the photo throughput of digital satellites, it's fun to think about what could be built with modern technologies. But that's probably best saved for another post.
Check it out! This is the baffle for the R7 StreetView camera. The final apertures are being cut with a laser. These apertures have to line up with the view cone of the cameras fairly tightly, in order to clip off a ghost reflection in the lens that I missed during optical design. The ghost only hits the sensor when the sun is in a 5 degree wide band just outside the field of view. I suspect it's a fairly complex internal reflection path, since a simple double bounce ghost ought to sweep through the center of the lens as the sun does.
I had the idea of welding the whole baffle together and then as a final operation cutting the apertures, with the hope that the metal would stop moving around once we'd cut smaller apertures and finished welding. It didn't really work out perfectly. There was a fair bit of tolerance stackup between the internal lenses and this external baffle. Also, the baffle mounted to 8 screws, and those 8 points would tend to get significantly out of alignment, so that the baffle shape that you ended up with depended on what order you tightened the screws. As a result we had to open up some extra margin on the baffle openings. I know for a fact that the sun can leak into the corners. After a quick check, I can't find any examples of the ghost, though, so the baffle is doing a decent job.
R7 at work:
R7 at play. Hi Ryan! Note the ears on top. Those are the twin GPS antennae... the way I intended all R7s to be.
The
clear stuff coming out of your kettle spout is 100% vaporized water.
(I'll sidestep pedantic discussions of the meanings of "steam" and
"water vapor".) It quickly entrains the cooler air around the kettle
and mixes with it, forming a mixing cloud, which you can see and most
folks refer to as "steam".
The
mixing cloud forms because the relationship between the vapor pressure
of water and temperature is not linear but rather curved. It's the red
set of dots above. This is really important, so take a moment with the
idea. As the temperature of the water in the kettle rises from 20 C
(room temperature) to 100 C (boiling), the pressure component of the
air/vapor mix above it due to the vapor rises from something quite tiny
to 1 bar (standard atmospheric pressure).
Now let's dilute that
ejected 100% water vapor with some dry air at 20 C. The mixed result is
cooler than 100 C and has a lower vapor pressure than 1 bar (because
not all the stuff is water vapor any more). Once you've accounted for
the different heat capacities of water vapor and water, and the thermal
expansion or contraction of the constituents, you get the blue curve
above. You read the curve like so: if I mix some amount of 20 C dry air
into some 100 C water vapor, so that the result is 60 C, then it's
partial pressure of water vapor is 0.29 bar. At least, that would be
the partial pressure if all the water was in vapor form.
The
really neat thing here is that the blue curve is ABOVE the red curve all
the way down to 26 degrees C, which happens to correspond to 95.8% dry
air and 4.2% water vapor. At any mixing ratio between this and 100% water vapor, there
will be more water in the mixing cloud than can be in vapor phase, and
some will be in the form of very tiny (micron) water droplets. However,
once the mixing cloud is sufficiently diluted with dry air, there will
be less water vapor in it than the maximum that can be in vapor phase,
and all those tiny droplets with their huge surface air to volume ratio
will immediately evaporate.
What this means is that the mixing
cloud grows to 23 times bigger than the pure water vapor expelled from
the kettle before it disappears.
Someone recently asked me if aerial survey could be accomplished with a swarm of tiny drone aircraft. Imagine foot-long planes made of styrofoam, carrying the camera, IMU, and GPS of a cellphone, minus the display, with a far larger battery for propulsion. How can this fail to be a cheaper way to survey large areas?
I've worked on a drone program before, but that was with a gas-powered UAV weighing 25 kg empty with a 2 kg payload and a range of around 1100 km over an 6 hour flight, costing in excess of $50k. Operational costs were expected to be dominated by the cost of crashing. I can say confidently that these are not a cheap way to survey anything.
In researching this blog post, I learned about solar-electric RC airplanes, which are a completely different kettle of fish. The Aphelion, above, was built for under $500 (materials only) in Professor Mark Drela's MIT lab, and can fly at 5-6 m/s for as long as the sun is 6.5 degrees above the horizon, which is something like 8 to 12 hours a day. That puts the daily range around 150 km.
Crucially, the whole airplane weighs 1300 grams distributed over a 3.3 meter wingspan. My guess is that a Cessna might survive an in-flight impact with one of these, and maybe even something a bit heavier.
A survey drone version would be ruggedized for e.g. 1000 flights MTBF, would carry a 200 g camera payload and a 50g battery, would fly at 12 m/s, and cover 300 km during a day's flight. Higher speeds lead to more coverage and better survivability in higher winds, but also smaller wings and less solar power. The battery allows flight with intermittent solar coverage, due to clouds and building blockages.
This range is not great. If a drone maintained 10 m/s from 9AM to 3PM, it would cover 216 km. For comparison, a King Air B200, which is something of a standard twin-turboprop airplane for aerial survey, flies at 140 to 150 m/s and can cover 1700 km in a morning.
These drones are going to require an ultralight camera -- basically a cellphone camera. The camera scales surprisingly well, primarily because you are comparing cellphone sensors to huge custom CCDs used in e.g. Leica ADS80 aerial cameras, and of course the cellphone sensors are far better because they have far more engineering investment. Nowadays I would use an Aptina MT9F002 14-megapixel sensor with a 10mm lens, which will deliver 2 inch resolution from 1000 feet up. This will deliver a swath of over 200 meters. Because the drone will bounce around a fair bit, we'll need a very generous overlap between the imagery captured in one flight line and the next, so the flight pitch will be something like 150 meters. For comparison, an ADS80 camera in a B200 flies with a 1 kilometer flight pitch when shooting this resolution.
Due to the close range and excellent sensor there is no need for a gimbal for the drone camera. I'd use a 400 microsecond exposure time. Combine that with the 140 microradian pixel field of view, and the camera can tolerate rolls and pitches of 25 to 50 degrees per second without excessive motion blur. I think even a little drone with very low wing loading can deliver that most of the time. If we shoot 94% forward overlap between frames (1 frame per second), which is possible because we are going so slowly, then if we lose a few shots due to motion blur we can afford to just drop them. We'll collect about 100 GB of data in 6 hours at that frame rate, which will be storable on a microSD card next year.
Now we come to the very bad news for this idea. The combination of low speed and small flight pitch means that we'll need something like 100 drones to replace a ADS80 in a B200. If you talk to R/C folks, you will find that they spend at least as much time fixing their aircraft as they do flying them. So those 100 drones are going to require an army of people to launch, recover, and repair them.
The bottom line is that using drones instead of manned aircraft still seems like a bad idea... if you can use a manned aircraft. But here is how survey drones might turn out to be a really wonderful idea: If you can make the drone light enough that it just won't hurt anyone when it crashes, then you might be able to get permission to fly down city streets. THAT would rock, because you'd simultaneously solve a bunch of problems that a bunch of folks (Nokia/Navteq, TomTom/TeleAltas, Google) are spending hundreds of millions of dollars a year on today:
You would be collecting street level imagery without having to wait for traffic. Much, much faster.
The native perspective of the cameras would be the one that people actually want. This is just my intuition, but my sense from working on Street View is that it was too damn slow and difficult to navigate. I think people want a raised perspective, but just not raised too much.
You would be collecting the sides of buildings at high resolution. Right now there is a major gap between street-level resolution (Google's Street View is 3mm resolution) and aerial resolution (Google's 45 degree view is 100mm resolution). That gap means that the sides of buildings don't match up well, which is a problem if, like Apple and Google, you want to deliver great looking 3D models of buildings.
Aerial obliques don't actually work in places like New York because the buildings are too tall compared to the street width. However, if you fly between the buildings, this is no longer a problem! This is a major issue now that over half the world's population (likely the wealthier half) live in cities.
I think it's a big stretch to imagine getting permission to fly your UAVs down city streets, and technically it would be a big challenge just to plan the routes... you'd probably end up needing to survey the city from high altitude first. It sure is fun to think about, though.
The Department of Homeland Security has said that it wants imagery delivered in the cloud. Several counties and states have expressed the same desire. Amazon S3 prices are quite reasonable for final imagery data delivered, especially compared to the outrageous prices that imagery vendors have been charging for what amounts to a well-backed-up on-site web server with static content. I've heard of hundreds of thousands of dollars for serving tens of terabytes.
Everyone wants the reliability, size, and performance scalability of cloud storage. No matter how popular your imagery becomes (e.g. you happen to have the only pre-Sandy imagery of Ocean City), and no matter how crappy your local internet link happens to be, folks all around the world can see your imagery. And many customers are more confident in Amazon or Google backups than their local IT backups.
But everyone also wants the reliability of local storage. E911 services have to stay up even when the internet connection goes down. So they need their own local storage. This also helps avoid some big bandwidth bills on the one site you know is going to hammer the server all the time.
So really, imagery customers want their data in both places. But that presents a small problem because you do want to ensure that the data on both stores is the same. This is the cache consistency problem. If you have many writers frequently updating your data and need transaction semantics, this problem forces an expensive solution. But, if like most imagery consumers you have an imagery database which is updated every couple of months by one of a few vendors, with no contention and no need for transaction semantics, then you don't need an expensive solution.
NetApp has a solution for this problem which involves TWO seriously expensive pieces of NetApp hardware, one at the customer site and one in a solo site with a fat pipe to Amazon. The two NetApp machines keep their data stores synchronized, and Amazon's elastic cloud accesses data stored at the colo over the fat pipe. This is... not actually cloud storage. This is really the kind of expensive shoehorned solution that probably pisses off customers more than me because they have to write the checks.
The right answer (for infrequently-updated imagery) is probably a few local servers with separate UPSes running Ceph and something like rsync to keep updates synchronized to S3. Clients in the call center fail over from the local servers to S3, clients outside the call center just use S3 directly.
I feel sure there must be some Linux vendor who would be happy to ship the Nth system they've build to do exactly this, for a reasonable markup to the underlying hardware.
Continuing with the recent theme of analyzing both natural and man made cameras, I thought I'd analyze an insect eye in this post. In previous posts I've analyzed a gigapixel airborne security video camera and the eye of a raptor. I'm working on analyses of the F-35's Distributed Aperture System as well as a really gigantic film camera for future posts.
The picture above is of a blue-spotted Hawker dragonfly. This creature apparently catches its prey in midflight, so it's supposed to have big, high resolution eyes. According to John Patterson in this Discover magazine article, it has 28,000 lenses in each eye. Judging from the picture, those eyes are probably covering half of the possible 4*pi solid angle, so each tiny lens is covering a solid angle of 112 microsteradians. That would be Low Resolution to my mind. Critters like spiders and grasshoppers, with even smaller eyes, must be commensurately worse.
To give some context, a human in bright light has a pixel field of view of 25 nanosteradians. You can see across your yard (70 feet) what the dragonfly can see from 1 foot. If it is going after an 8mm long house fly, it needs to close within 80 cm in order to get a full pixel on the target. At that range the house fly can probably hear the dragonfly, although I haven't analyzed that and insect hearing might also be much worse than human hearing. This dragonfly is not a long-range hunter like a bird, but rather an opportunist. I very much doubt it can pull off a high-relative-speed attack like a Peregrine falcon, which requires good angular resolution and gimbal stabilization.
Interestingly, the creature collects about the same amount of light per pixel that the human eye does. The entire creature is about 70mm long. Judging from the picture below, the eyes are probably 10 mm across. That means those little lenses are 53 microns across. On a bright day outside, your human pupils constrict to 3 mm, 55 times larger. But since the dragonfly's pixel field of view is 70 times bigger, it actually receives about 50% more light per pixel than you do.
I used to be concerned that the tiny dragonfly lenses would cause diffraction blurring. Supposing for a moment that they look at green light, their Rayleigh criterion is about 12 milliradians, nicely matched to their pixel angular field of view of 10.5 milliradians. If the effective pupil of each lens is substantially smaller than the allotted space (low fill factor), then the dragonfly would have to use a shorter wavelength to avoid diffraction problems. It doesn't look like a big problem.
I don't think the dragonfly has to stabilize the scenery with it's neck. A dragonfly flying at 1 meter/second sees scenery 50 cm away to the side going by at 2 radians/second. To get less than 3 pixels of motion smear, it would have to integrate the exposure for less than 15 milliseconds. That's just a bit quick, so the creature probably sees some motion blur to the side, but not so much in front of it, since there is considerably less angular motion in the direction of flight.
Here's an explanation from RED about why high frame rate video is a good thing. They are right, and this is a great explanation with really impressive examples.
Here's an exotic high flying camera: ARGUS-IS. I've been trying to figure what these folks have been up to for years, and today I found an SPIE paper they published on the thing. What follows is a summary and my guesses for some of the undocumented details. [Updated 30-Jan-2013, to incorporate new info from a Nova documentary and a closer reading of the SPIE paper.]
Vexcel Ultracams also use four cameras
with interleaved sensors
It looks like BAE is the main contractor. They've subcontracted some of the software, probably the land station stuff, to ObjectVideo. BAE employs Yiannis Antoniades, who appears to be the main system architect. The lenses were subcontracted out to yet another unnamed vendor, and I suspect the electronics were too.
Field of view: 62 degrees. Implied by altitude 6 km and object diameter 7.2 km.
Image sensor: 4 cameras, each has 92 Aptina MT9P031 5 megapixel sensors. The paper has a typo claiming MT9P301, but no such sensor exists. The MT9P031 is a nice sensor, we used it on the R5 and R7 Street View cameras.
It's easy to interface with, has high performance (quantum efficiency over 30%, 4 electrons readout noise, 7000 electrons well capacity), and is (or was) easy to buy from Digikey. (Try getting a Sony or Omnivision sensor in small quantities.)
Focal length: 85mm. Implied by 2.2 micron pixel, altitude of 6 km, GSD of 15 cm. Focal plane diameter is 102mm. The lens must resolve about 1.7 gigapixels. I must say that two separate calculations suggest that the focal length is actually 88mm, but I don't believe it, since they would have negative sensor overlap if they did that.
F/#: 3.5 to 4. There is talk of upgrading this system to 3.7 gigapixels, probably by upgrading the sensor to the Aptina MT9J003. An f/4.0 lens has an Airy disk diameter of 5.3 microns, and it's probably okay for the pixels to be 2.2 microns. But 1.66 micron pixels won't get much more information from an f/4.0 lens. So, either the lens is already faster than f/4.0, or they are going to upgrade the lens as well as the sensors.
The reason to use four cameras is the same as the Vexcel Ultracam XP: the array of sensors on the focal plane cannot cover the entire field of view of the lens. So, instead, they use a rectangular array of sensors, spaced closely enough so that the gaps between their active areas are smaller than the active areas. By the way guys (Vexcel and ObjectVideo), you don't need four cameras to do this problem, it can be solved with three (the patent just expired on 15-Jul-2012). You will still need to mount bare die.
The four cameras are pointed in exactly the same direction. Offsetting the lenses by one sensor's width reduces the required lens field of view by 2.86 degrees, to about 59 degrees. That's not much help. And, you have to deal with the nominal distortion between the lenses. Lining up the optical axes means the nominal distortion has no effect on alignment between sensors, which I'm sure is a relief.
The sensor pattern shown in the paper has 105 sensors per camera, and at one point they mention 398 total sensors. The first may be an earlier configuration and the latter is probably a typo. I think the correct number is 92 sensors per camera, 368 total. So I think the actual pattern is a 12x9 rectangular grid with 11.33mm x 8.50mm centers. 16 corner sensors (but not 4 in each corner) are missing from the 9x12=108 rectangle, to get to 92 sensors per focal plane. The smallest package that those sensors come in is 10mm x 10mm, which won't fit on the 8.5mm center-to-center spacing, so that implies they are mounting bare die to the focal plane structure.
They are carefully timing the rolling shutters of the sensors so that all the rolling shutters in each row are synchronized, and each row starts it's shutter right as the previous row finishes. This is important, because otherwise when the camera rotates around the optical axis they will get coverage gaps on the ground. I think there is a prior version of this camera called Gorgon Stare which didn't get this rolling shutter synchronization right, because there are reports of "floating black triangles" in the imagery, which is consistent with what you would see on the outside of the turn if all the rolling shutters were fired simultaneously while the camera was rotating. Even so, I'm disappointed that the section on electronics doesn't mention how they globally synchronize those rolling shutters, which can be an irritatingly difficult problem.
They are storing some of the data to laptop disk drives with 160 GB of storage. It appears they may have 32 of these drives, in which case they've got enough space to potentially store the entire video stream, but only with very lossy video compression. The design presented has only JPEG2000 (not video) compression, which will be good for stepping through the frames, but the compression ratio will be bulky enough that there is no way they are storing all the video.
They have 184 FPGAs at the focal plane for local sensor control, timestamping, and serialization of the data onto 3.3 Gb/s fiber optics. Supposedly the 3.3 Gb/s SerDes is on the FPGA, which sounds like a Virtex-5 20T. But something is odd here, because having the SerDes on the FPGA forces them to choose a fairly beefy FPGA, but then they hardly do anything with it: the document even suggests that multiplexing the two sensor data streams, as well as serialization of those streams, happens outside the FPGA (another typo?). So what's left for a Virtex-5 to do with a pair of sensors? For comparison, I paired one Spartan-3 3400A with each sensor in R7, and we were able to handle 15 fps compression as well as storage to and simultaneous retrieval from 32 GB of SLC flash, in that little FPGA. Maybe the SerDes is on some other device, and the FPGA is more of a PLD.
The data flows over fiber optics to a pile of 32 6U single board computers, each of which has two mezzanine cards with a Virtex 5 FPGA and two JPEG2000 compressors on it.
Now here's my critique of this system design:
They pushed a lot of complexity into the lens.
It's a wide angle, telecentric lens. Telecentric means the chief rays coming out the back, heading to the focal plane, are going straight back, even at the edges of the focal plane. Said another way, when you look in the lens from the back, the bright exit pupil that you see appears to be at infinity. Bending the light around to do that requires extra elements. This looks a lot like the lenses used on the Leica ADS40/ADS80, which are also wide angle telecentric designs. The Leica design is forced into a wide angle telecentric because they want consistent colors across the focal plane, and they use dichroic filters to make their colors. The ARGUS-IS doesn't need consistent color and doesn't use dichroics... they ended up with a telecentric lens because their focal plane is flat. More on that below.
The focal lengths and distortions between the four lenses must be matched very, very closely. The usual specification for a lens focal length is +/- 1% of nominal. If the ARGUS-IS lens were built like that, the image registration at the edge of field would vary by +/- 500 microns. If my guesses are right, the ARGUS-IS focal plane appears to have 35x50 microns of overlap, so the focal lengths of the four lenses will have to match to within +/- 0.07%. Wow.
"The lenses are athermalized through the choice of glasses and barrel materials to maintain optical resolution and focus over the operational temperature range." Uh, sure. The R7 StreetView rosette has 15 5 megapixel cameras. Those lenses are athermalized over a 40 C temperature range, and it was easy as pie. We just told Zemax a few temperature points, assumed an isothermal aluminum barrel, and a small tweak to the design got us there. But those pixels have a field of view of 430 microradians, compared to the pixels behind the ARGUS-IS lens, which have a 25 microradian PFOV. MIL-STD-810G, test 520.3, specifies -40 C to 54 C as a typical operating temperature range for an aircraft equipment bay. If they had anything like this temperature range specified, I would guess that this athermalization requirement (nearly 100 degrees!) came close to sinking the project. The paper mentions environmental control within the payload, so hopefully things aren't as bad as MIL-STD-810G.
The lenses have to be pressure compensated somehow, because the index of refraction of air changes significantly at lower pressures. This is really hard, since glasses, being less compressible than air, don't change their refractive indices as fast as air. I have no particularly good ideas how to do it, other than to relax the other requirements so that the lens guys have a fighting chance with this one. Maybe the camera can be specified to only focus properly over a restricted range of altitudes, like 4km to 8km. (ARGUS-IR specifies 0 to 10km. It's likely ARGUS-IS is the same, so no luck there.) Or maybe everything behind their big flat window is pressurized.
They made what I think is a classic system design mistake: they used FPGAs to glue together a bunch of specialized components (SerDes, JPEG compressors, single board computers), instead of simply getting the job done inside the FPGAs themselves. This stems from fear of the complexity of implementing things like compression. I've seen other folks do exactly the same thing. Oftentimes writing the interface to a off-the-shelf component, like a compressor or an encryption engine, is just as large as writing the equivalent functionality. They mention that each Virtex-5 on the SBC has two 0.6 watt JPEG2000 chips attached. It probably burns 200 mW just talking to those chips. It seems to me that Virtex could probably do JPEG2000 on 80 Mpix/s in less than 1.4 watts. Our Spartan-3 did DPCM on 90+ Mpix/s, along with a number of other things, all in less than 1 watt.
I think I remember reading that the original RFP for this system had the idea that it would store all the video shot while airborne, and allow the folks on the ground to peruse forward and backward in time. This is totally achievable, but not with limited power using an array of single-board PCs.
Let me explain how they ended up with a telecentric lens. A natural 85mm focal length lens would have an exit pupil 85mm from the center of the focal plane. Combine that with a flat focal plane and sensors that accept an f/1.8 beam cone (and no offset microlenses), and you get something like the following picture. The rectangle on the left is the lens, looking from the side. The left face of the right rectangle is the focal plane. The big triangle is the light cone from the exit pupil to a point at the edge of the focal plane, and the little triangle is the light cone that the sensor accepts. Note that the sensor won't accept the light from the exit pupil -- that's bad.
There are two ways to fix this problem. One way is to make the lens telecentric, which pushes the exit pupil infinitely far away from the focal plane. If you do that, the light cone from the exit pupil arrives everywhere with it's chief ray (center of the light cone) orthogonal to the focal plane. This is what ARGUS-IS and ADS-80 do.
The other way is to curve the focal plane (and rename it a Petzval surface to avoid the oxymoron of a curved focal plane). Your retina is curved behind the lens in your eye, for example. Cellphone camera designers are now looking at curving their focal planes, but it's pretty hard with one piece of silicon. The focal plane array in ARGUS-IS is made of many small sensors, so it can be piecewise curved. The sensors are 7.12 mm diagonally, and the sag of a 85 mm radius sphere across 7.12 mm is 74 microns. The +/- 9 micron focus budget won't allow that, so curving the ARGUS-IS focal plane isn't going to allow a natural exit pupil. The best you can do is curve the focal plane with a radius of 360 mm, getting 3.6 mm of sag, and push the exit pupil out to about 180 mm. It's generally going to be easier to design and build a lens with an exit pupil at 2x focal length rather than telecentric, but I don't know how much easier. Anyway, the result looks like this:
As I said, the ARGUS-IS designers didn't bother with this, but instead left the focal plane flat and pushed the exit pupil to infinity. It's a solution, but it's not the one I would have chosen.
Here's what I would have done to respond to the original RFP at the time. Note that I've given this about two hours' thought, so I might be off a bit:
I'd have the lenses and sensors sitting inside an airtight can with a thermoelectric cooler to a heat sink with a variable speed fan, and I'd use that control to hold the can interior to between 30 and 40 C (toward the top of the temperature range), or maybe even tighter. I might put a heater on the inside of the window with a thermostat to keep the inside surface isothermal to the lens. I know, you're thinking that a thermoelectric cooler is horribly inefficient, but they pump 3 watts for every watt consumed when you are pumping heat across a level. The reason for the thermoelectric heat pump isn't to get the sensor cold, it's to get tight control. The sensors burn about 600 mW each, so I'm pumping 250 watts outs with maybe 100 watts.
I'd use a few more sensors and get the sensor overlap up to 0.25mm, which means +/-0.5% focal length is acceptable. I designed R5 and R7 with too little overlap between sensors and regretted it when we went to volume production. (See Jason, you were right, I was wrong, and I've learned.)
Focal plane is 9 x 13 sensors on 10.9 x 8.1 mm centers. Total diameter: 105mm. This adds 32 sensors, so we're up to an even 400 sensors.
Exiting the back of the fine gimbal would be something like 100 flex circuits carrying the signals from the sensors.
Hook up each sensor to a Spartan-3A 3400. Nowadays I'd use an Aptina AR0330 connected to a Spartan-6, but back then the MT9P001 and Spartan-3A was a good choice.
I'd have each FPGA connected directly to 32GB of SLC flash in 8 TSOPs, and a 32-bit LPDDR DRAM, just like we did in R7. That's 5 bytes per pixel of memory bandwidth, which is plenty for video compression.
I'd connect a bunch of those FPGAs, let's say 8, to another FPGA which connects to gigabit ethernet, all on one board, just like we did in R7. This is a low power way to get connectivity to everything. I'd need 12 of those boards per focal plane. This all goes in the gimbal. The 48 boards, and their power and timing control are mounted to the coarse gimbal, and the lenses and sensors are mounted to the fine gimbal.
Since this is a military project, and goes on a helicopter, I would invoke my fear of connectors and vibration, and I'd have all 9 FPGAs, plus the 8 sensors, mounted on a single rigid/flex circuit. One end goes on the focal plane inside the fine gimbal and the other goes on the coarse gimbal, and in between it's flexible.
I'd connect all 52 boards together with a backplane that included a gigabit ethernet switch. No cables -- all the gigE runs are on 50 ohm differential pairs on the board. I'd run a single shielded CAT-6 to the chopper's avionics bay. No fiber optics. They're really neat, but power hungry. Maybe you are thinking that I'll never get 274 megabits/second for the Common Data Link through that single gigE. My experience is otherwise: FPGAs will happily run a gigE with minimum interpacket gap forever, without a hiccup. Cheap gigE switches can switch fine at full rate but have problems when they fill their buffers. These problems are fixed by having the FPGAs round-robin arbitrate between themselves with signals across that backplane. Voila, no bandwidth problem.
The local FPGA does real time video compression directly into the flash. The transmission compression target isn't all that incredible: 1 bit per pixel for video. That gets 63 channels of 640x400x15 frames/sec into 274 Mb/s. The flash should give 1 hour of storage at that rate. If we want 10 hours of storage, that's 0.1 bits/pixel, which will require more serious video compression. I think it's still doable in that FPGA, but it will be challenging. In a modern Spartan-6 this is duck soup.
The computer tells the local FPGAs how to configure the sensors, and what bits of video to retrieve. The FPGAs send the data to the computer, which gathers it up for the common data link and hands it off.
I'll make a guess of 2 watts per sensor+FPGA+flash, or 736 watts. Add the central computer and switch and we're at 1 kilowatt. Making the FPGAs work hard with 0.1 bit/pixel video compression might add another 400 watts, at most.
No SSDs, no RAID, no JPEG compression chips, no multiplexors, no fiber optic drivers, no high speed SerDes, no arrays of multicore X86 CPUs. That's easily half the electronics complexity, gone.
UPDATE 25-Jan-2013: Nova ran a program on 23-Jan-2013 (Rise of the Drones) which talks about ARGUS-IS. They present Yiannis Antoniades of BAE systems as the inventor, which suggests I have the relationship between BAE and ObjectVideo wrong in my description above. They also say something stupid about a million terabytes of data per mission, which is BS: if the camera runs for 16 hours the 368 sensors generate 2,000 terabytes of raw data.
They also say that the ARGUS-IS stores the entire flight's worth of data. I don't think they're doing that at 12 hertz, certainly not on 160 GB drives. They've got 32 laptop drives in the system (one per single board computer). If those store 300 GB apiece, that's 10 terabytes of total storage. 16 hours of storage would require 0.05 bits/pixel -- no way without actual video compression. The JPEG2000 compressor chips are more likely to deliver at best 0.2 bits/pixel, which means they might be storing one of every four frames.
UPDATE 27-Jan-2013: An alert reader (thanks mgc!) sent in this article from the April/May 2011 edition of Science and Technology Review, which is the Lawrence Livermore National Laboratory's own magazine. It has a bunch of helpful hints, including this non-color-balanced picture from ARGUS-IS which lets you see the 368 sensor array that they ended up with. It is indeed a 24 x 18 array with 16 sensors missing from each corner, just as I had hypothesized.
The article mentions something else as well: the Persistics software appears to do some kind of super-resolution by combining information from multiple video frames of the same nearly static scene. They didn't mention the other two big benefits of such a scheme: dynamic range improvement and noise reduction (hence better compression). With software like this, the system can benefit from increasing the focal plane to 3.8 gigapixels by using the new sensor with 1.66 micron pixels. As I said above, if the lens is f/3.5 to f/4.0 lens they won't get any more spatial frequency information out of it with the smaller pixels, but they will pick up phase information. Combine that with some smart super-resolution software and they ought to be able to find smaller details. Question though: why not just go to the MT9F002, which gives you 14 million 1.4 micron pixels? This is a really nice, fast sensor -- I've used it myself.
The article also mentions 1000:1 video compression. That's very good: for comparison, H.264 level 4 compresses 60 megapixels/second of HDTV into 20 megabits/second, which is 0.33 bits/pixel or 36:1 compression. This isn't a great comparison, though, because Persistics runs on almost completely static content and H.264 has to deal with action movie sequences. In any case, I think the Persistics compression is being used to archive ARGUS-IS flight data. I don't think they are using this compression in the aircraft.
