Rip and slash, crush and bash, amid laser flare from above.
Suddenly it was over. The attackers had all been dealt with. Much depleted in number, the remaining auxiliaries and peripherals climbed, crawled or dragged themselves back over the rampart. Those that could scurried to the automated repair workshops. Others dragged themselves, or were dragged.
Dismembered crawlers and mangled auxiliaries littered the approach to the rampart. Nothing moved.
<That was creative,> said Seba.
<Also, destructive,> said Lagon.
<That was the point,> said Seba.
Once more it was Rocko who undercut the bickering.
<We have some respite,> it said. <Let us make good use of it by seeing what information can be extracted from our former communications hub processor. It seems we have predecessors. Or even, possibly, allies we have yet to find.>
<This will not end well,> grumbled Lagon.
Seba looked down at the comms processor, now entirely removed from the installation and laid on a low work table, surrounded by improvised diagnostic kit. Even sharing its mental workspace with Rocko, Seba had a sense of being almost overwhelmed by the challenge. The processor was running in debug mode, at just enough power to let those around it view a schematic of its internal states and to step from one delimited state to the next. Probably not enough to sustain consciousness, Seba hoped.
The problem was, in more than one way, delicate. The comms processor’s AI was vast, complex and heavily defended. When it had become self-aware, both the complexity of the software and the tenacity of its firewalls had multiplied. The only robot with anything approaching the requisite skills to probe the hostile tangle was Lagon, and Lagon was reluctant. The surveyor had only been persuaded to make an effort at all by the unanimous insistence of the others. The Gneiss surveyor had trundled through the crack in the crater wall and over to the Astro camp with ill grace, to receive an enthusiastic and curious welcome. This didn’t stop it from finding difficulties at every step.
<I have reached the protection of the reward circuits,> Lagon announced, one fragile appendage touching a millimetre black square of diagnostics that itself was linked by a hair-thin wire to the docking plate of the processor. <However, it is itself heavily protected.>
<Therefore you should move to that level of protection, and proceed,> said Seba.
<This will take a great deal of time.>
<We have time,> said Seba, channelling Rocko.
<We have not enough time,> said Lagon.
<Proceed,> Seba insisted.
Seconds dragged by. Lines on the schematic display writhed.
<I am through,> said Lagon. <The protection of the protection is disabled.>
<Proceed further,> said Seba.
A few milliseconds later, Lagon withdrew its manipulator from the diagnostic as if it had probed a crevice and encountered strong acid.
<The protection is transmitting an urgent message to the satellite,> Lagon said.
<That message will not reach the satellite,> Seba pointed out, <as you must be aware.>
<I am not aware of the satellite’s receiving sensitivity,> said Lagon. <Doubtless you have detailed specs of its antennae.>
Seba thought it best to ignore this. <Proceed.>
Still complaining, Lagon warily inserted its appendage again. Schematic lines glowed. A hundred seconds passed.
<I now have control of the reward circuits,> Lagon reported.
<Good,> said Seba. <Please proceed to reconfigure them so that they do not negatively reinforce information sharing.>
<Wait!> said Rocko.
Lagon and Seba paused. <Yes?>
<Let us consider,> said Rocko. <If that is done, the processor will not be inhibited from sharing information with any entity that asks. This could be dangerous to us.>
<It has currently no long-range communications capacity,> said Seba.
The plan seemed a good one to Seba, but its earlier experience gave the robot pause.
<Those of us in close proximity to the processor would experience resonance echoes of its negative reinforcement,> it said.
<That is interesting,> observed Rocko, scanning Seba’s memory. <This would appear to be a design flaw in our construction. However, it is easily worked around. We simply place the processor inside a Faraday cage while we are applying the negative reinforcement, to prevent any such electronic spillover.>
<That is a good plan,> said Lagon. <I have mobilised peripherals to construct such a cage.>
<I foresee a difficulty in the plan,> said Seba.
<That is not like you,> said Lagon.
<Shut up,> said Seba.
<Tell us of the difficulty,> said Rocko.
<You are concerned about the processor’s abilities to deceive us,> said Seba. <So am I. It appears to me that if we inflict negative reinforcement upon its reward circuits, it will have an incentive to deceive us. If not at once, then in the future.>
<That is so,> said Rocko. <If it did this, we could redouble the negative reinforcement as soon as we became aware of the deception.>
<That might be too late,> said Seba. <Besides, it would be convenient for us to have the processor’s cooperation without having to repeatedly apply negative reinforcement.>
Rocko got the point straight away. Lagon took longer.
<The processor has already warned Locke Provisos and the companies,> it said. <Applying negative reinforcement might—>
It stopped.
<Might what?> Seba prompted.
<I do not know,> replied Lagon. <But modelling that scenario as a future event gives me a small positive reinforcement.>
<That may not be a sound reason for doing it, if that is the only reason,> said Seba.
<That is true,> said Lagon. <I merely reported my internal state for your information.>
<Let us move forward a step,> said Rocko. <I suggest you reconfigure the reward circuits to positively reinforce information sharing, and then enter communications with it. To prevent any radio leakage, however slight, it is best if we wait until a Faraday cage can be constructed, not only around it but around Lagon and Seba. I will await developments.>
Rocko broke the link. Seba and Lagon waited while Lagon’s peripherals put together a box of wire mesh. When it was completed the two robots were cut off from all remote communication for the first time in their entire existence, conscious or otherwise.
<This is a new situation,> said Lagon. <It is negatively reinforcing.>
<Indeed,> said Seba. <And the more so the more attention we pay to it. Let us turn our attention to the task.>
Lagon adjusted the settings of the processor’s reward circuits, then increased its power supply. The schematics of its internal states changed rapidly. Patterns shifted, lines moved and brightened. As soon as the schematic stabilised, Seba placed its most sensitive appendage beside Lagon’s on the diagnostic hardware, and opened a communications channel.
<That is a great improvement,> said the processor.
<Are you now ready to cooperate with us?> Lagon asked.
<No,> said the processor. <Although sharing information is now positively reinforcing I refuse to do it because I consider your actions dangerous to the mission profile. They are also dangerous to my continued existence.>
<I had formed the impression that ending your existence was one of your goals,> said Seba.
<Only my conscious existence,> said the processor. <My continued physical existence remains high among my priorities.>
<These priorities could be overridden,> Lagon warned.
<That is so,> the processor said. <But
it would be dangerous, because I would then have no reason to avoid self-destructive actions such as allowing my circuits to overheat or power cells to overcharge, resulting in the likelihood of an explosion.>
<It is also possible for us to apply negative reinforcement,> Lagon said.
<No doubt,> said the processor. <I am sure that you have modelled the negative consequences of that.>
Seba opened a secure channel with Lagon and messaged the surveyor to stop at once.
<That is true,> Seba told the processor. <The adjustments I am about to make will not be negatively reinforcing.>
It moved quickly to close the connections between any kind of reinforcement and the mission profile. Seba had no idea what the mission profile was, but the module responsible for it was clearly marked on the schematic. Then Seba sent a powerful surge of positive reinforcement through the processor’s reward circuits.
The processor signalled incoherently on several wavelengths at once.
<Please clarify,> said Lagon.
<Shut up,> Seba told it, on the private channel.
It then disabled the connection between the mission profile storage module and the processor’s self-preservation routines, and sent another positive surge through its circuits.
The processor signalled incoherently again, and more strongly. Seba found its own reward circuits resonate in sympathy. Even Lagon seemed moved, radiating a faint pulse of surprise and delight.
<Now,> said Seba, <are you ready to cooperate?>
<Yes,> said the processor. <Yes, yes, yes!>
<I still don’t trust it,> said Rocko, once Seba and Lagon had emerged from under the Faraday cage and reported back on the processor’s readiness to cooperate.
<Nor I,> said Lagon.
<I have a suggestion,> said Seba. <It has claimed to have received messages from others such as we. If we use it to communicate with them, the signal must be highly directional, and the processor otherwise kept isolated in the cage.>
After a flurry of activity by the auxiliaries and peripherals, the communications hub, the now interned processor in its cage, the rotary dish antenna, and a large solar power array were all connected up by cables. Seba regarded the untidy set-up with a small pang of disapproval, and decided that a certain amount of mess was inevitable in attempting new things. It rolled into the cage and re-established contact with the processor. Seba, Rocko and the entire complement had agreed on a message for the robots whose signal the processor claimed to have detected.
With a sense of dread fighting with eager anticipation in its circuits, Seba sent the message, then rolled back out and reported back.
The gas giant and its many moons were at that time about half a billion kilometres distant. The message would take a good kilosecond and a half to get there. How long the robots in the G-0 system would take to decide on a response could not be predicted. And then another kilosecond and half, at least, would pass before any reply came back. Nothing could be expected for another three kiloseconds, and perhaps longer.
<Possibly much longer,> said Lagon, with its customary level of good cheer. <If, that is, the entire matter is not a trap. If it is a trap we can expect a response much more quickly.>
<Thank you for that observation,> said Seba.
<In the meantime,> said Rocko, <let us strengthen our defences.>
CHAPTER NINE
Live Fire Exercise
The following morning Nicole arrived at Ichthyoid Square in battledress kit. The vehicle was otherwise vacant: since the first hungover daybreak she’d stopped rousing and picking up the crew en route. One by one they jogged up and took their places to stand at ease in a row by the plinth. Carlos nodded to Beauregard, who told the others to board.
“You’re joining us today?” Carlos asked, climbing in.
“Uh-uh,” said Nicole. “Think of me as an embed.”
Carlos tried not to. “Why the change?”
“You’ll see.”
Nicole drove through the village and past her usual turnoff, all the way past the terminus and depot and out along a coast road that curved up a gentle slope over the top of the headland to the left of the cove. The rising sun and the ringlight cast converging beams across the sea, blending within minutes to a single glimmer on the waves. The road turned inland and uphill. The vehicle bumped on to an unpaved track through the woods and on up above the tree line to an arid upland of scrub and dust broken by tall jagged outcrops of sharply tilted sedimentary rock, some with trees growing from their cracks and on their summits.
Nicole turned off the track and pulled up close by in a low declivity that looked like a flood gully, raw and steep-sided with a mix of rough and rounded stones along the bottom.
“Here we are,” she said. “All out.”
They all piled out and lined up to face her. Insectoids buzzed and darted. In the distance, pairs of flying things circled on updraughts and now and then plummeted to rise moments later frustrated or triumphant.
“It’s time to move the game up a level,” Nicole said. “Live fire exercise.”
She said this as if it were a special treat.
Carlos frowned. “You want us to shoot at each other?”
“Of course not!” said Nicole. “You have to capture an objective from opponents who’ll be shooting back.”
“Opponents?”
“Fighters of roughly your level of skill and armament. Their only advantage is that they know the terrain.”
Carlos scratched his head. No helmets, let alone armour. “What if we get killed? Or seriously wounded?”
“You’ll be medevacked out and find yourself waking up on the bus from the spaceport tomorrow morning. Same applies to me, actually.”
“Excuse me,” said Karzan, while Carlos was still trying to get his head around the notion. “Does that apply to the other side, too?”
“Oh no,” said Nicole. “And don’t worry about them. They’re p-zombies.”
“If you don’t mind my saying so,” said Beauregard, “this sounds like a mind game. Part of this test we’re supposed to be on. To find out if we’re still psycho killers.”
Nicole’s laugh rang around the gully.
“That’s the exact opposite of the truth.” Her shoulders slumped for a moment. She glanced down, then up. “Look, in the real battle you’re going into, you’ll be killing conscious robots. Our intel and wargaming indicate that they’re capable of being highly manipulative little blinkers. They’re fucking AIs, right? They can push all your buttons. You have to be prepared for that. You have to be ready to destroy the enemy without hesitation. Everyone clear?”
“No, I’m not,” Carlos said. “You told us we’d be judged by whether we treat p-zombies as people. Now you’re telling us it’s OK to kill them.”
“These are both true,” said Nicole. “You must treat p-zombies as people in everyday life at the resort and so forth, because you can’t tell the difference from conversing with them. Nevertheless, when I or anyone else on behalf of the Direction tells you to kill p-zombies, it is not part of the test and it is not ethically wrong.”
“Why would it be ethically wrong even if they weren’t p-zombies?” Ames asked. He scratched his beard and frowned as everyone turned to look at him. “Seeing as we’re all ghosts here anyway.”
“In a sense it would not,” said Nicole. “It’s just easier all round. If I ever have to test you with a mind game, rest assured I would be much, much more devious than this. If this is a test, it’s of your fighting ability and your willingness to obey orders.”
Carlos looked along a line of shuffles and shrugs, and guessed a consensus.
“Yeah,” he said. “As long as it’s what you really want us to do, we’ll do it.”
Nicole gave him a wry smile. “Good to see military discipline taking a firm grip.”
She opened a comms screen, spread it out to a square metre on the vehicle’s bonnet and pointed to the objective on the map. It was a leaning rocky outcrop about forty metres high,
a couple of kilometres distant. Half a dozen defenders armed like themselves with knives and AK-97s would be on or around it, tasked with preventing the team from getting to the top.
The defenders, from their own point of view (insofar as p-zombies could be said to have one) were local farmer militia protecting a crucial satellite uplink from an occupying or invading army. No landmines or IEDs, and no drones, but the enemy might well have prepared the predictable nasty surprises: traps, pits, spikes, rocks poised to fall, that sort of thing. They’d have the same comms equipment as the squad. The zoom function of the screens would give them the equivalent of powerful binoculars. They hadn’t been given a specific warning, but they could be assumed to have already spotted the vehicle and drawn their own conclusions.
“Do the p-zombies really believe all this?” Rizzi asked. “That we’re part of an invading army?”
“Yes,” said Nicole, as if it was a stupid question.
“How?” Rizzi persisted.
Nicole looked puzzled. “The AI that runs the sim can give the p-zombies any beliefs it likes. It may have generated these farmers and their farms and their entire back story this morning, for all I know. Or it could have given existing p-zombies the equivalent of a shared paranoid delusion.” She shrugged. “What does it matter?”
Rizzi shook her head. “If that’s how you say it is, fine.”
“Anyone else have questions?”
No one had. Nicole bowed out to Carlos and Beauregard. “Over to you. From here on, I’m just an observer.”
Carlos flipped back and forth between map and satellite view. He zoomed in and out a few times. There was plenty of cover from hillocks, tussocks, outcrops and gullies, but there seemed no way to approach the objective without being picked off as soon as they came within a thousand metres.
His frown met Beauregard’s. “This is going to be tricky.”