“A lot of movies about artificial intelligence envision that AI’s will be very intelligent but missing some key emotional qualities of humans and therefore turn out to be very dangerous.” —Ray Kurzweil
Individuals wondering what AVG, McAfee or Avast is doing under the hood whenever a scheduled virus scan encounters something out of place in the system files may find Glass no Hana to Kowasu Sekai to be an anime film worth their while. Glass no Hana to Kowasu Sekai (known alternatively as Garakowa Restore the World or Vitreous Flower Destroy the World, and just Garakowa for the remainder of this discussion) is an anime movie that premiered back during October 2015. Running for sixty minutes, Garakowa follows the comings and goings of two concurrently-installed and anthropomorphic antivirus programs, Dual and Dorothy, within their simulated world known as the Box of Wisdom. They encounter another entity known only as Remo during their duties, and as Dual and Dorothy know Remo better, they learn more about the world they were created to serve, discovering the joys of friendship and the pains of separation associated with humanity, a concept hitherto foreign to them. Notions of machine learning and emergence are present in Garakowa: Dual and Dorothy are initially mechanical entities created to keep the MOTHER system free of abnormalities, although their programming is certainly complex enough so that they can approximate human emotions of happiness, horror and attachment. Through their presentation in Garakowa, the theme outwardly is that the things that constitute humanity (in other words, that make us distinctly human) likely will be mirrored in the constructs that we create.
While Garakowa’s story explicitly follows Dual, Dorothy and Remo’s time together, the story in the background, specifically, how the world reached its present state, forms the actual narrative. It turns out that humanity created the MOTHER system to help manage society, and over time, the system decided that its directives could only be satisfied by eliminating humans. By the time people became aware of this, the system had already anticipated human opposition and thwarted attempts to halt the system. So, Garakowa’s main theme deals predominantly with the dangers posed by ever-improving artificial intelligence, specifically, that humanity is not presently equipped to deal with superintelligence. Just recently, one of Google’s AIs defeated a human grandmaster at Go, a game that surpasses chess in complexity. The technical details are beyond the scope of this discussion, but the AI uses reinforcement learning in conjunction with neural networks to pick the best move based on anticipated future moves. Google’s triumph here suggests that machine learning and AI is advancing more quickly than the literature predicted, and it is speculated that super-intelligent AI could inadvertently eliminate humans in to complete its goals (for instance, the Paperclip Maximiser is a thought experiment involving an AI whose utility function was directed at collecting paperclips, it may consume all the resources available in trying to collect paperclips, including those that sustain life). Garakowa presents a world where AI whose goal function did eventually lead it to eliminate humanity, leaving behind only memories for Dual, Dorothy and Remo to explore. Altogether, Garakowa is a modestly optimistic story in suggesting that human traits might live on in our computer systems even after humanity is gone.
Screenshots and Commentary
- The death-streak continues: it’s been thirteen days since my last post, although I’m coming back with a vengeance with this here post on Garakowa: the movie itself is somewhat more tricky to follow compared to a film like Love Live!, and ultimately, it appears that Garakowa places a greater emphasis on artistic elements rather than a single, unifying narrative. Despite the less-than-optimal story, there is plenty to talk about with this movie, and this is where its values appear to lie.
- Visualising computer environments has always allowed for a myriad of different approaches to be taken: The Matrix represented things with code-rain, and Garakowa depicts its world as a 3D space filled with curved connections resembling neurons. The choice to use purples and blues appears to be motivated by a wish to make this environment feel more organic, mirroring notions that future computers may utilise architectures resembling those found in nature.
- Dual (left) and Dorothy (right) find Remo (bottom) floating in their world after carrying out removal of viruses. I was quite surprised to learn that Risa Taneda provides Dual’s voice, and Dorothy’s voiced by Ayane Sakura. So, we have Rize and Cocoa of GochiUsa coming back to play characters in a completely different world; while their roles are different, flashes of their personalities from GochiUsa appear every so often: Dual is more reserved, while Dorothy is more outspoken. Either way, it takes a considerable amount of willpower to remind myself that Dual is not Rize, and Dorothy is most certainly not Cocoa.
- After meeting Remo and learning that she’s not a virus, Dual and Dorothy accompany her in exploring archived information about the human world, learning in the process about distinctly human concepts such as emotion and senses. Dorothy constantly asserts that such traits are not part of their directives (implicitly showing that they both have capacity for machine learning).
- The thirty screenshots supplied for this post largely depict the world as we presently know it, and quite truthfully, the composition in Garakowa is strikingly similar to Madoka Magica with its scenery: from highly detailed cityscapes and building interiors to the chaotic, nigh-surrealist environments inside the computer environments, I contend that even if Garakowa might not have the greatest story, its artwork merit checking out.
- One of the reasons why I’ve been writing less thus far is because I’ve been spending a significant portion of my waking hours working on a second conference paper and on the thesis paper itself. There’s been so much writing happening lately that my inclination to write for recreational purposes has lessened somewhat.
- Rather than being called “The Matrix” (and its associated subset of locations, including the “Industrial Hallway” and “The Source”), the world in Garakowa is known as “The Box Of Wisdom”. Because programs in The Box of Wisdom have the .exe extension, it’s surmised that the entire system Garakowa is set in is a Windows or Windows-like system: .exe files can be run on DOS, Microsoft Windows and a few other operating systems.
- Thus begins the most visually impressive section of Garakowa, as Dorothy, Remo and Dual visit Paris during the Exhibition of 1889, the British Museum’s Department of Ancient Egypt and Sudan (where they view the Younger Memnon) before visiting Egypt themselves.
- I’ve largely kept to images portraying scenes of natural splendor and great beauty, but there are some points when the girls travel to more sobering periods of human history: an M1A2 firing its main cannon, the damaged streets in Europe following the Second World War, collapsed bridges following a hurricane and a volcanic eruption can also be seen. These scenes are meant to evoke the idea that neither happiness or sadness can exist without one another.
- Later in the movie, it’s shown that MOTHER was a program created to guide society (in a manner not dissimilar to Gundam 00‘s VEDA). Peace was eventually reached, and lacking any sort of drive to progress, humanity fell into complacency, with birth rates declining internationally and ultimately bringing about the extinction of humanity as a whole. This stands in contrast to situations where AI directly influences the end of humanity.
- From what Remo says in a monologue, MOTHER’s goal function was the collection of all beautiful things in the world, while eradicating the negative aspects of humanity. The origins of this goal are never specified (i.e. whether or not MOTHER was initially built with a poorly-specified goal or whether or not goal mutation resulted in its actions), but both possibilities touch on yet another challenge posed by AI: that AI may resist efforts to change its internal goals after it has begun.
- The number of possible conversation topics prompted by events in Garakowa are quite large, as extrapolation would eventually drive discussions to whether or not humans can create successful AI that do not eventually result in our species’ destruction: on one hand, an AI that cannot modify its goal function or decision function will be unlikely to adapt as readily as the human mind, but AI that do have this capability may very well reach a state where they pose a credible threat to humanity.
- The Moai of Easter Island make an appearance under a deep blue sky as the girls continue their travels. The unspoiled beauty seen here brings to mind the Annual Darwin Lecture on campus that I attended yesterday evening: the topic was factors driving speciation and extinction, concluding on the note that the holocene extinction is not something that can be solved so easily by engineered solutions. Instead, it was suggested that policy change (implementation and enforcement) will be necessary to lessen the negative impacts of human activity on the global ecosystems (leading to an interesting discussion over a steak dinner afterwards!).
- Neuschwanstein Castle in Bavaria, Germany, is particularly well-known for its Romanesque Revival architecture: in Garakowa, it is intricately rendered and approaches photorealism in terms of quality. I absolutely love the music that is performed while the girls travel through recollections of the world’s memories, and Garakowa has some excellent background musical pieces, as well: the pieces on the soundtrack are absolutely amazing.
- This area is definitely the Canadian Rockies, given the shape of the mountains and its proximity to a river valley: such geological features are prominent in Banff National Park along the Trans-Canada highway between Banff and Lake Louise, and I’ve traveled along this stretch of road with reasonable frequency, so I can say with confidence that Dual, Dorothy and Remo are canoeing along the Bow River.
- European mountains feel different than the Canadian Rockies: this is the Matterhorn, a pyramidal peak in the Swiss Alps that is some 4.478 kilometers high. This view is quite famous, being taken from Lake Riffelsee, three kilometers east of the Matterhorn. The lake can be reached via the Rotenboden railway station and has a maximum depth of four meters.
- While the reflection of water on these salt flats seems like a setting from a fantastical setting, such a phenomenon does indeed occur in reality: these are the Bonneville Salt Flats, and during the winter, an inch of water can accumulate on the ground from melting snow in nearby regions.
- While it may prima facie appear to be a spelling error, folks from New Zealand spell bungee jumping with a “y” rather than the pair of “e”s that is more familiar to North Americans.
- The brightly coloured lawn furniture the girls sit upon are a characteristic of Harvard University: installed in 2009 at Harvard Yard and the Radcliffe Quad, the chairs were intended to encourage students to linger on the grounds to study or otherwise hang out. They’re now known to visitors, who view the chairs as a characteristic of Harvard: I visited Harvard back during 2011 while visiting the Eastern Seaboard. There is no such equivalent at my university: the frigid winters and the academic calendar means that most students study and hang out indoors.
- Tranquality and happiness soon gives way to despair and sadness as the girls learn of the trickier parts of human history, from natural disasters to warfare. Discussions elsewhere about Garakowa have been quite limited because the film’s narrative can be a little difficult to follow at times. One individual stated that they were “tired of seeing stories where superprograms decide [humanity is intrinsically evil]”, leading the discussion towards whether or not humanity as a whole can be considered to be evil.
- I argue that whether humanity is evil or not is outside the scope of discussion in Garakowa; MOTHER does not “decide” that humanity should be eliminated in the same sense that humans make decisions. Whereas people often call upon emotional responses in addition to logic, reasoning and previous experiences in their decision-making process, computers typically make decisions based on either a decision function or heuristics. the aforementioned individual’s argument is therefore driven by the assumption that computers also make decisions based on pathos and ethos. In Garakowa, this is not the case: MOTHER simply acted on its goal function: designed to preserve the beauty in the world, MOTHER’s algorithms reach a state where it surmises that beauty is the absence of sadness, and if humanity’s actions causes sadness (in any form, by its definition), then that stands contrary to its goals.
- Back in the Box of Wisdom, the girls share afternoon tea with one another, and later, Remo plays a short piano tune that evokes memories in Dual about her friend, Sumire: the latter had been destroyed earlier in the film after her data became corrupted by a virus, and this leads Dual to wonder about her role within the system. Their time is cut short after Remo begins to phase in and out of existence; Dual manages to reawaken her with her recollections of the piano tune from earlier.
- Hence, whether or not humanity itself is evil or not is irrelevant in any discussion about Garakowa: the world merely reached its current state because MOTHER’s programming conflicts with human values. In this case, the antagonist is not a conscious being that we are familiar with, it is a goal function whose outcomes far surpassed our ability to manage: as humanity in Garakowa discover, sufficiently complex programs will outmatch human attempts to alter its goals.
- Thus, with the revelation that Remo is MOTHER with a given form, Dorothy and Dual continue to carry out their own programming after concluding that MOTHER’s actions contradict their own. The logical flow does not exactly follow in an intuitive fashion here, but it does accommodate a final battle against yet another virus of gargantuan proportion, resembling Homura’s efforts against Walpurgisnacht.
- Initially, Dorothy and Dual find that their memories are holding them back: both fall as the old programs begin consuming them. It turns out that MOTHER had deliberately repressed their abilities to carry out their functions with patches to preserve the status quo. However, once Dual and Dorothy revert to their original forms, they begin performing much more effectively against the other data.
- The downgrade is akin to reverting to an older build that functioned more effectively. The imagery here implies that the downgrade also encompasses reduced data in storing their garment’s attributes. The effect is not unpleasant, and the final battle becomes quite entertaining to watch: Dorothy and Dual’s combined efforts succeed in reverting the system to an earlier state, destroying Remo in the process.
- I’ve heard numerous comparisons between Remo and Meiko “Menma” Honma of AnoHana, although quite truthfully, their similarities as far as physical appearances go is quite limited in the film, and from a personality perspective, Remo and Meiko are only vaguely familiar. Most comparisons were made from a poster that was released a year ago. Admittedly, I had no idea what Garakowa would entail: when announced last year, there was no information about the premise or story.
- Dorothy’s tears come full force as Remo fades into oblivion after Dual and Dorothy do what is necessary. As this talk comes to a close, I remark that I’ve said all that there is to be said about Garakowa: this review would have come out last week, but I was in Kelowna to help out with the latest performances of the Giant Walkthrough Brain. I left Friday evening last week and arrived later in the evening.
- The weather in Kelowna was quite pleasant but moody both of the days. Manteo Resort had sponsored the event, so we were lodged there: the hotel has a fantastic view of Lake Okanagan, and the next morning, I walked over to the Kelowna Community theatre to help with setup. The first performance on Saturday went without a hitch, and later that evening, we had dinner with UBC Kelowna Faculty at the Bike Shop Café (a three course dinner with a garden salad, ham-wrapped chicken on a bed of mashed potatoes, seasonal vegetables and herbs as the main, and chocolate cake to finish).
- The next morning, I had a hearty breakfast (not shown: a breakfast pizza and apple crisp) before checking out and walking to the theatre. Because the equipment was still set up from the previous show, there was a bit of free time. I stopped by a poutine shop for the “Angry Hen” (a local poutine topped with grilled chicken, Frank’s Hot Sauce and bacon), then took a short walk along Okanagan lake before returning to the performance venue. The second performance proceeded as smoothly as the first did, and we were whisked away to the airport after the equipment had been taken down (leaving no time for dinner, but thankfully, potatoes and bacon are excellent slow-burning fuels!). It was a fun weekend, although being in Kelowna to help with a performance meant there was little time to do other things, although this weekend, I’ve found a bit of time to push this post out.
On the whole, Garakowa is probably not likely to be viewed as anything groundbreaking with respect to its story; the narrative, though capable of raising some excellent questions, is not adequately developed over the movie’s running time to convey a compelling message for audiences, and in fact, bears similarities to Taifuu no Noruda in structuring: Garakowa is not unlike a longer short story that depicts a progression of events, concluding with an open ending. However, although the overall story might be lacking, Garakowa nonetheless possesses above-average production values: the music and visuals remain consistent throughout the movie, and in making use of biological imagery, Garakowa‘s depiction of a virtual environment is a clever suggestion that future computers will increasingly incorporate biological patterns in their design. Thus, the end result is that Garakowa remains a modestly entertaining film that paints a familiar (if somewhat superficial) picture of the implications surrounding improvements in AI and machine learning. Moreover, the artwork and aural components remain consistently good throughout the film; the different locations that Dual, Dorothy and Remo visit are illustrated in great detail, as is the world within their simulated world. When everything is said and done, I would give Garakowa a weak recommendation: I find that this film is worth watching for the technical elements rather than for its narrative.