bunnie

While open source is necessary for trustable hardware, it is far from sufficient. This is because “hashing” hardware – verifying its construction down to the transistor level – is typically a destructive process, so trust in hardware is a massive time-of-check/time-of-use (TOCTOU) problem. This talk helps us understand the nature of the TOCTOU problem by providing a brief overview of the supply chain security problem and various classes of hardware implants. We then shift gears to talk about ways to potentially close the TOCTOU gap, concluding with a curated set of verifiable components that we are sharing as an open source mobile communications platform – a kind of combination hardware and software distribution – that we hope can be useful for developing and deploying all manner of open platforms that require a higher level of trust and security.

The inconvenient truth is that open source hardware is precisely as trustworthy as closed source hardware. The availability of design source only enables us to agree that the designer’s intent can be trusted and is likely correct, but there is no essential link between the hardware design source and the piece of hardware on your desk. Thus while open source is necessary for trustable hardware, it is far from sufficient. This is quite opposite from the case of open source software thanks to projects like Reproducible Builds, where binaries can be loaded in-memory and cryptographically verified and independently reproduced to ensure a match to the complete and corresponding source of a particular build prior to execution, thus establishing a robust link between the executable and the source. Unfortunately, “hashing” hardware – verifying its construction down to the transistor level – is typically a destructive process, so trust in hardware is a massive time-of-check/time-of-use (TOCTOU) problem. Even if you thoroughly inspect the design source, the factory could modify the design. Even if you audit the factory, the courier delivering the hardware to your desk could insert an implant. Even if you carried the hardware from the factory to your desk, an “evil maid” could modify your machine. This creates an existential crisis for trust – how can we know our secrets are safe if the very hardware we use to compute them could be readily tainted? This talk addresses the elephant in the room by helping us understand the nature of the TOCTOU problem by providing a brief overview of the supply chain security problem and various classes of hardware implants. We then shift gears to talk about ways to potentially close the TOCTOU gap. When thinking about hardening a system against supply chain attacks, every component – from the CPU to the keyboard to the LCD – must be considered in order to defend against implanted screen grabbers and key loggers. At every level, a trade-off exists between complexity and the feasibility of non-destructive end-user verification with minimal tooling: a system simple enough to be readily verified will not have the equivalent compute power or features of a smartphone. However, we believe that a verifiable system should have adequate performance for a select range of tasks that include text chats, cryptocurrency wallets, and voice calls. Certain high-risk individuals such as politicians, journalists, executives, whistleblowers, and activists may be willing to use a device that forgoes bells and whistles in exchange for privacy and security. With this in mind, the Betrusted project brings together a curated set of verifiable components as an open source mobile communications platform - a combination open source hardware and software distribution. We are sharing Betrusted with the community in the hopes that others may adopt it as a reference design for developing and deploying all manner of open platforms that require a higher level of trust and security.

Other people: Sean "xobs" Cross Tom Marble

This talk will present a historical narrative of the background behind how the NeTV + Milkymist inspire the HDMI2USB then helped the NeTV2 projects and how they all became interlinked through events like Congress! From the study of this history, we will attempt to distill a few core lessons learned that can hopefully be applied to other open hardware projects.

Open hardware projects tend to evolve differently from open software projects. Even though it’s very easy to fork an open software project, pull requests and merges help ensure the main branch of a project continues to improve. Furthermore, open software projects tend to evolve along with their tools, as evidenced by the concurrent maturation of Servo and Rust, or Linux and Git. In contrast, open hardware projects tend to fork and then fracture the community as they gain commercial success and go closed, as evidenced in the evolution of the 3D printer and drone communities. There are also few examples of hardware projects that co-evolve with their tools.

The evolution of the NeTV + Milkymist to the HDMI2USB to the NeTV2, along with the concurrent maturation and adoption of the Migen and LiteX ecosystem, is one of these rare examples.

This talk will present a historical narrative of the background behind each of these projects and how they became interlinked. From the study of this history, we will attempt to distill a few core lessons learned that can hopefully be applied to other open hardware projects.

One important lesson is that open anything takes time and patience, and so the project itself needs to be in a space where it can move at an appropriate pace to grow a healthy community while maintaining relevance against potentially better-funded closed-source options.

Another lesson is the importance of crowdfunding as a mechanism to marshall community around a given release, as opposed to relying upon sources of financing that expect direct returns on investment (such as venture capital or loans).

A final example of a lesson we will discuss is the importance of picking the right tools to co-evolve with the project. While many open source options alternatives exist to closed-source tools, it’s problematic if the design tools somehow limit the hardware implementation or introduces flaws in the design itself -- and hardware, unlike software, cannot be patched. Migen/LiteX are front-end tooling for describing FPGA designs (“gateware”), and their output being a bitstream means it can be in the hardware space while enjoying the privilege of easy patching and updates. Furthermore, Migen/LiteX offer critical features and performance metrics unavailable in the closed-source alternatives, meaning that these are not simply design tools, they are also core to enabling the competitiveness of the hardware itself.

Related previous Chaos Congress Talks

• Latest developments around the Milkymist System-on-Chip - A roundup of one the most advanced open hardware projects
• Implementation of MITM Attack on HDCP-Secured Links - A non-copyright circumventing application of the HDCP master key
• Dissecting HDMI - Developing open, FPGA-based capture hardware for conference & user group recording

Presenters

• Tim 'mithro' Ansell is the founder of TimVideos. He is known for having to many projects including hardware like the HDMI2USB.tv, I'm Tomu and many more!
• Andrew 'bunnie' Huang is a hacker, activist and author who is known for many projects including hardware like the NeTV, Chibitronics, Novena, and Chumby.

Other people: Tim 'mithro' Ansell

The participation of women in computer education is low; undergraduate classrooms in Germany were only 10% female in 2000[1]. The picture at the primary school level is fuzzier, as students do not declare majors at that level, but evidence indicates the trend starts from a young age. Can we make computer education more gender-inclusive? Presenting technology in familiar, non-threatening contexts can lead to more balanced gender participation. For example, Chibitronics uses the context of papercraft to present electronics to beginners; the familiarity of papercraft improves the participation of women of all ages in the creation of electronics. Based on these learnings, we have devised the “Love to Code” platform, an open source hardware-to-cloud stack which combines the familiarity of paper craft with a web-based, driver-free embedded firmware development environment based on FSK audio provisioning via a headphone jack. In this talk, we will dive into the novel open source technical contributions of this platform, which includes the audio-based provisioning protocol and the unique rigid-flex design of the circuitry enabling papercraft integration, as well as the multi-threaded client OS and cloud infrastructure required to complete the picture. This combination of new technology with familiar interfaces aims to lower the barrier to computer education, thus making coding a more accessible and inclusive activity.

Computer technology tends to be a male-dominated field. One study from 2002 placed female participation in undergraduate computer education classrooms around 10% for Germany, and 26% for the US[1]. The picture is fuzzier at the primary school level, because students do not declare majors at such a young age, but evidence indicates that this strong gender bias has roots extending to primary school. Can we make computer education more inclusive? There is evidence that presenting technology in familiar, non-threatening contexts can lead to more inclusive participation. As an example, Chibitronics uses the context of paper craft to present electronics to beginners. Paper craft – the art of manipulating and coating paper as exemplified by origami or painting – has universal accessibility among almost all ages, genders, and ethnic groups. In our approach, novices start with the familiarity of paper, and combine it with copper tape and electronic stickers to build circuits. Demographic analysis of the Chibitronics user base indicates this approach is succeeding at gender inclusivity: over 70% of individual buyers are female. Follow-up interviews indicate that Chibitronics acts as a gateway to more advance projects and skills. After bootstrapping into basic electronics, users pick up skills such as soldering to build larger and more permanent projects. We would like to continue this trend beyond circuit crafting and into the realm of embedded firmware coding. Creating parallel and series combinations of LEDs and switches can only go so far; users rapidly outgrow the basic techniques and want to add patterns and interactivity to their projects through coding. Coding for the web is as simple as learning a text editor and pointing a browser at a URL, but coding for an embedded hardware target, such as an Arduino, inevitably requires platform-specific drivers, which can require installation and debugging. The problem is further exacerbated by the fact that primary school educators rarely possess the skillset to install, debug, and maintain a classroom of computers for embedded development. Furthermore, the waning popularity of laptops and desktops means the most inclusive platforms are now mobile devices such as smartphones and tablets, even though their touchscreen keyboards are not ideal for extensive programming. In order to tackle the problem of inclusivity, we make embedded firmware development nearly universal by coupling a cloud-based editing and compilation infrastructure with an audio-based firmware upload protocol suitable for implementation on low-cost microcontrollers. Cloud-based editing and compilation eliminates the barrier of downloading and running a C compiler on the client, while the audio-based firmware upload protocol allows virtually any computer, smartphone, or tablet to communicate with target hardware implementing our demodulator. We demonstrate this end-to-end solution via the Chibitronics “Love To Code” (LtC) open source hardware platform. Users code for their LtC boards using Arduino-flavored C++ as a baseline language by visiting our demo site (either https://chibitronics.com/ltc (CodeMirror editor) or https://chibitronics.com/cb (Codebender/ACE editor)). Users enter their code on the client-side editor, which is then uploaded as plain text to a cloud-based C++ compiler and returned as an object code binary. The binary is then packetized, coded and modulated using browser-side Javascript into an FSK audio stream with a baud rate of 8000 bps. The frequency plan was chosen to minimize interaction with audio “enhancement” filters found in devices such as iPhones, and to enable the raw audio stream to be recorded as a 128kbps CBR MP3 with minimal distortion. This allows pre-compiled binaries to be stored as songs; one could even entertain the thought of cutting these songs into vinyl records and provisioning microcontrollers using nothing more than an analog turntable. The FSK audio stream is coupled from the headphone jack into the microcontroller PCB via a modified microUSB cable. The USB interface is used only for power; the data pins are inert during the provisioning process. Audio arrives at the LtC hardware via the “fifth pin” in the microUSB plug that is normally reserved for identifying A or B type devices. This hack allows users to power programmed LtC devices using any regular microUSB cable. Within the LtC device, the audio signal is DC coupled and level-shifted using a passive RC network into a single ADC pin on an NXP CPU, the MKL02Z32VFK4. This 48 MHz Cortex M0+ features 32k of FLASH and 4k of RAM and costs under $1 in modest volumes, less than one third the cost of the AVR used in the Arduino Leonardo. The microcontroller uses noncoherent demodulation to recover the bitstream. The received data is hash-checked for integrity and, if it passes, is committed to FLASH memory. Since our code is open source, one can add FSK audio provisioning to a wide range of microcontrollers with built-in ADCs for the price of a couple resistors and a capacitor. Because microphone jack pin assignments are non-standard between different device ecosystems, we designed the protocol to operate with only forward error correction through trivially redundant transmission of packets. In other words, we cannot rely on a feedback path being available for the device to request a retransmission of a corrupt packet, due to a lack of standardization among microphone interfaces. Fortunately, in our application, the signal to noise ratio is typically quite good and most errors are due to burst noise, e.g. a notification tone generated by another application during device programming. In this scenario, simple hamming codes (such as SECDED) do little to improve robustness of the protocol, and more advanced error correction mechanisms such as BCH codes requires more computational power and storage space than available in our inexpensive target device. As a result, we handle error correction through trivial replication of data by playing the song three times over. In addition to the audio firmware downloader code, the target CPU also runs an implementation of the open source, multi-threaded RTOS ChibiOS (no relationship to Chibitronics despite the Japanese-derived “chibi” prefix). We expose the threading API so that advanced users can create multi-threaded applications on LtC. This is an advantage over the popular Arduino platform, which has no native support for multi-threading. In order to reduce code upload times over an 8000bps link to within the “impatience limit” of an 8-year old, we pre-load the LtC device with common shared library routines, including floating point, string manipulation, a WS2812 RGB LED driver, and primitives for a low speed USB stack. Because of the library pre-load, we are able to reduce the upload time of most simple programs to under two seconds. The built-in libraries, audio demodulation framework, and multi-threaded OS consumes 22k of memory, leaving 10k for user application code. Learning from our prior experiences with paper electronics, the physical design of the LtC hardware borrows from familiar and non-threatening interaction paradigms. The LtC circuit board uses a rigi-flex construction: the rigid portion allows for mechanically robust connectors and compatibility with fine-pitch QFN packages, while the flex portion enables seamless integration with paper-based circuit techniques. As a result, users can integrate the LtC hardware into their projects using a variety of techniques, from sticker-like methods to an electronic clipboard paradigm. The electronic clipboard method is well-suited to classroom environments, as it combines unlimited re-use of the LtC hardware with low-cost, quickly customized and paper-craftable circuitry. This combination enables meaningful learning engagements within an hour of two of classroom time, at a price point accessible by non-elite, publicly funded institutions. Finally, the entire framework is licensed under an open source license. Our hope is that once a novice is hooked on coding, they can go as deep as they want, perhaps even remixing our hardware and firmware frameworks into new and creative applications we never had never dreamed of. The Chibitronics Love to Code platform is familiar: users with little or no prior technical background find themselves immediately in familiar territory thanks to the papercraft interaction design. It is easy to use: cloud compilation coupled with audio firmware upload ensures compatibility with a broad range of devices from laptops to smartphones. It is accessible: the combination of cost-optimized core technology with paper as a consumable substrate reduces the barrier of access for schools on tight budgets. The net result is an embedded coding framework for inclusive computer education. [1] V. Galpin, SIGSCE Bulletin, Vol 34 No 2, 2002 June. “Women in Computing Around the World”

We introduce Fernvale, a reverse-engineered, open hardware and software platform based upon Mediatek's MT6260 value phone SoC. The MT6260 is the chip that powers many of the $10 GSM feature phones produced by the Shanzhai. Fernvale is made available as open-licensed schematics, board layouts, and an RTOS based upon the BSD-licensed NuttX, as well as a suite of open tools for code development and firmware upload. We discuss our technical reverse engineering efforts, as well as our methodology to lawfully import IP from the Shanzhai ecosystem into the Maker ecosystem. We hope to establish a repeatable, if not labor-intensive, model for opening up previously closed IP of interest, thereby outlining a path to leveling the playing field for lawful Makers.

There is a set of technology which Makers are legally allowed access, and there is a much larger set of technology which is used to make our every day gadgets. Access to the best closed-source technology is prevented via barriers such as copyright (limiting your ability to learn how it works), patent (limiting your ability to make something similar), and supply-chain (limiting your ability to buy it). As a result, open-licensed, Maker-friendly technologies have trailed closed-source technology in terms of cost, performance, and features. Makers operating under Western IP law are legally bound by these barriers, and are forced to settle for Arduinos, Beaglebones, Raspberry Pis and Novenae. However, all of these are a far cry in terms of cost, performance, and features from what consumers typically expect from boxes purchased in retail stores. Our research into the Chinese ecosystem indicates there is another way. Originally marginalized as outlaws and copycats, the Shanzhai of China – China's counterpart to the Western hacker-maker – exist in a realm where copyright and patent barriers are permeable, a state which we refer to as 'gongkai'. As a result, knowledge and access to state of the art closed source technology has diffused into the Shanzhai ecosystem. Today, they have moved beyond the rote copying of Nokia, Samsung, and Apple, and have created a thriving, vibrant ecosystem where mobile technology is rip/mix/burned; their products are mass-produced at a rate of millions per month for the “rest of the world”, e.g. emerging markets such as Africa, Brazil, India, Indonesia, and Russia. About a year ago, we did a tear-down of an example $12 phone, and contrasted it to the Arduino Uno. For $29, the Arduino Uno gets you a 16MHz, 8-bit CPU with 2.5k of RAM, and USB plus a smattering of GPIO as the sole interfaces. For $12, a phone out of the Chinese gongkai ecosystem gets you a 260 MHz, 32-bit CPU with 8MiB of RAM, with USB, microSD, SIM, quad-band GSM, Bluetooth, an OLED display and a battery. It begs the question of why, when Makers talk about IoT technologies in the West, they typically think of wifi-powered solutions in the $20-70 range, versus a GSM platform in the $10-$20 range. In this lecture, we disclose an attempt to short-circuit the disclosure barrier. We are releasing an open hardware and software solution built around the Mediatek MT6260. The MT6260 is a 32-bit ARM7EJ-S SoC with 8MiB of PSRAM in-package, as well as USB, LCD, touchscreen, audio, Bluetooth, quad-band GSM, dual-SIM, FM radio, UART, keypad, SD card, camera, and other peripherals integrated. The chip can be purchased on the over-the-counter market for about $2-3 in China. We call our solution built around this chip “Fernvale”. Fernvale is similar to the “LinkIt ONE” recently released by Mediatek and Seeed Studios, based upon the MT2502A SoC and targeted at IoT and wearables. LinkIt indicates a new direction for Mediatek and we are optimistic that their effort indicates a new pattern of openness toward Makers. At the time of this proposal's submission, the details of the LinkIt ONE platform are still unfolding, but the basic feature set looks comparable to that of Fernvale. However, it seems the LinkIt SDK is still based upon a closed-source Nucleus RTOS providing services to an open Arduino-like API. Unlike LinkIt ONE, Fernvale runs a port of NuttX, a small-footprint BSD-licensed RTOS that is Posix and ANSI compliant, and includes a partial set of drivers for the available hardware peripherals. The mainboard is laid out to function as either a SoM (system on module) or as a truncated Arduino shield (with the appropriate headers populated), and focuses on the computational abilities of the platform. In other words, Fernvale is not positioned as a mobile phone solution per se, but rather as an Engineering Development Kit (EDK) for embedded applications that can benefit from a highly-integrated, low-cost high-performance microcontroller solution such as the MT6260. As a result, the mainboard breaks out a selection of GPIO as well as the speaker, battery, USB, and SD card interfaces. The mainboard also serves as a base platform for rallying a larger community of developers who can aid with the task of reverse engineering and writing legally open drivers for its massive peripheral set. Two expansion headers are provided on the mainboard. A larger UX header can be used to attach a keypad + LCD + audio interface, for applications that require UI elements. A smaller analog header enables users to attach an RF front-end of their choosing, which could potentially enable GSM-compatible voice and data services, if drivers were to exist. This lecture will also discuss our experiences reverse engineering, and our approach to open-sourcing the MT6260. We had to reverse engineer significant portions of the system, including but not limited to circuit board layouts, hardware configuration options, bootloader protocols, partial register maps, and the internal boot ROM of the SoC. This reverse engineering effort was necessary to create a blob-free software implementation, and to give developers an alternative to Mediatek's proprietary firmware flashing utilities to upload code. It was also necessary to create schematics and circuit board maskworks which have an original copyright thereby giving us the right to pick an open license for the hardware designs. We took special pains to ensure our method was lawful and the resulting work is copyright-clean under U.S. law. We did review some non-open-licensed chip documentation and code examples available for download from open file-sharing sites. None of these materials were restricted by DRM. American copyright law contains a fair-use exception that allows limited copying and examination of such materials for the purpose of understanding the ideas and functional concepts embodied in them. We believe our download and review of those materials is fair use. Should potential copyright holders disagree with our interpretation, we invite any offended parties to engage us in rational discourse. We believe that Makers have for too long lived in the shadow of overbearing copyright laws. We need to develop an example of how to import ideas from less strict IP jurisdictions where innovation is flourishing; failing this, hardware Makers run the risk of being eternally behind the Shanzhai. Fernvale is our first attempt at developing a legal context for importing IP from the gongkai ecosystem into a fully open source solution; we hope our example will embolden other developers to pursue more ambitious targets. We also hope our work may, in the long term, catalyze meaningful Maker-friendly reform to Western IP law by raising awareness of the disparity between East and West, with the success of the Shanzhai serving as evidence of how permissive IP policy can be good for both grass-roots innovators (the Shanzhai) and big businesses (Mediatek and the phone network operators) alike.

Other people: Xobs

All “managed FLASH” devices, such as SD, microSD, and SSD, contain an embedded controller to assist with the complex tasks necessary to create an abstraction of reliable, contiguous storage out of FLASH silicon that is fundamentally unreliable and unpredictably fragmented. This controller is an attack surface of interest. First, the ability to modify the block allocation and erasure algorithms introduces the opportunity to perform various MITM attacks in a virtually undetectable fashion. Second, the controller itself is typically powerful, with performance around 50MIPS, yet with a cost of mere pennies, making it an interesting and possibly useful development target for other non-storage related purposes. Finally, understanding the inner workings of the controller enables opportunities for data recovery in cards that are thought to have been erased, or have been partially damaged. This talk demonstrates a method for reverse engineering and loading code into the microcontroller within a SD memory card.

TECHNICAL APPROACH Publicly available documentation on SD card controllers is scarce. However, based upon tear-down and decap analysis as well as a survey of the publicly available product briefs, most controllers are believed to be either an enhanced 8051, or an ARM derivative. A further challenge to overcome is the fact that SD card manufacturers typically reserve the right to change the controller IC within a card without updating the external markings to reflect the change. This policy favors the SD card manufacturers, as it allows them to swap out existing controllers for lower-cost devices as new controllers are introduced. However, it is problematic for users as it means that two otherwise identical looking cards can have different performance and/or bugs with which to contend. To kick off the effort, a survey of available cards was made at an SD card gray market in Shenzhen, China. Each card was dissected and visually inspected for cues, such as the layout of the traces going to the controller glob-top, that would indicate the type of controller within. About a dozen different controller types were identified, of which one was picked for further investigation due to its use of SLC FLASH memory. SLC is a good starting point for reverse engineering because no storage-level scrambling is required to prevent the read and write-disturb issues typical of MLC and TLC FLASH. A simple binary dump of the FLASH memory within the card revealed structure within the first erase block consistent with what we might expect for code storage. Since FLASH memory is inherently unreliable, four CRC + ECC protected copies of code are located within the first sector. This crude duplication scheme allows the card to boot even if bit errors creep into the code storage sector. We also noted the existence of the string “BUILDWIN” within the code storage sector, which indicates that the controller is likely from a series made by a company called Appotech. Product briefs from the Appotech/Buildwin websites indicated that the architecture of the code is likely an 8051-derivative, and the model of the controller is probably an AX211. At this point, our effort to reverse engineer the controller split into two paths. One path was static analysis, where extracted binaries and manufacturing-related tools were analyzed with IDA to determine key entry points, storage locations, and most importantly a method for injecting code into the card via the SD interface. The other path was dynamic analysis, where the signals going to the SD card bus and to the NAND FLASH were instrumented with logging and stimulus facilities, and the controller's operation could be observed with exquisite resolution, enabling a broad class of fuzzing and other brute-force analysis attacks, as well as the rapid confirmation or rejection of hypotheses generated by static analysis. Dynamic analysis was key in determining features such as the location of the GPIO control registers and the function and format of otherwise undocumented extended instruction opcodes. The static analysis path was assisted by the availability of official firmware burning routines, scavenged from Chinese language file-sharing websites. These are tools normally used during the production of SD cards, but made available on the gray market to enable (and correct) card capacity expansion fraud. Typically, these tools are used to flash an incorrect version of the firmware onto the SD cards, which would identify the card as having a much higher capacity than the physically available storage. This would allow unscrupulous dealers to sell, for example, aging 128MiB silicon as devices identifying themselves with 2GiB capacity. These bootleg tools would come with a collection of firmware blobs for loading onto the card, as well as a routine that communicates to a proprietary USB-based burning device. We did not have access to the burner, but static analysis of the communication protocol via code reverse engineering revealed that firmware loading is initiated through the application of a specific “secret knock” sequence during card boot. The dynamic analysis path was implemented using an embedded platform of our making, known publicly as Novena. It is a quad-core ARM CPU running Linux mated to a Spartan 6 FPGA with a high speed expansion port. The FPGA also has a 256 MiB DDR3 buffer slaved off of it. This buffer + FPGA was used to implement a combination logic analyzer + ROM emulator used to wrap the controller on the SD card. The ROM emulator presents a virtual, dual-ported NAND FLASH device to the SD card controller. The dual-port design allows us to modify and read the contents of the NAND FLASH on the fly, thereby allowing us to stimulate the controller IC with various fault conditions or code loops and measure its effect on the FLASH device. This tight coupling enabled us to rapidly discover, for example, which SFRs (Special Function Registers) are responsible for reading or writing to the Flash controller hardware. Furthermore, each port of the SD card controller is connected to a virtual logic analyzer within the FPGA that can store up to a 16MiB trace of transactions going to and from the controller from both the SD and FLASH buses. This very deep buffer length allowed us to observe behavior of the device from power-on to full operation, as well as observe stimulus-response loops to compound SD bus commands. SD INTERFACE OVERVIEW Before diving into the details of our approach, we review a few important aspects of the SD protocol. When an SD card first boots, it is running in SD/MMC mode. This is a 9-wire protocol, using the following pinout (note: pin 9 is located below and to the side of pin 1): 12345678 9|||||||\- DAT1 |||||||\-- DAT0 ||||||\--- GND |||||\---- Clock ||||\----- VDD +3.3V |||\------ GND ||\------- Command |\-------- DAT3 \--------- DAT2 SD commands consist of a start bit of 0, a sync bit of 1, a 6-bit command number (referred to as CMDn, where n is between 0 and 63,) four 8-bit arguments, a CRC7 checksum, and a stop-bit of 1. This yields a total of 48-bits (or 6 bytes) per command. While the card is processing the request, it will keep the Command line high, and will begin its communication with a start bit by bringing the Command line low, indicating "0". This will be followed by 38 bits of data, 7 bits of crc7, and a stop bit of "1", for a total of 48 bits. Bit values are sampled when the clock line is high, and may change when the clock line is low. Bytes are sent from high to low. For example, hex value 0x80 would be written out as 0b10000000. CMD0 must be sent before any other commands are processed. To send such a command, transmit the following six byes. {0x40 0x00 0x00 0x00 0x00 0x95} | | | | | \----- (CRC7 of ({0x40, 0x00, 0x00, 0x00}))<<1 | 1 | \----\----\----\---------- Four arguments of "0" \------------------------------ Start bit of 0, cmd bit of 1, CMD0) Then, continue to wiggle the clock, and look for the 48-bit response. Data is transmitted either on the DAT0 line (for 1-bit mode), or striped in parallel across DAT0-DAT3. As with the Command line, bit 7 is sent first, and bit 0 is sent last. In four-bit mode, DAT3 will send bit 7, DAT2 will send bit 6, DAT1 will send bit 5, and DAT4 will send bit 4. Bits 3-0 will be sent using DAT3-DAT0 on the next clock cycle. Data is transmitted with a start bit of 0, followed by the data (almost always 512 bytes), followed by a CRC16 of the data, and then a stop bit of 1. In the case of 4-bit mode, all four DAT lines send a start bit of 0, and each individual line sends its own CRC16 of the transmitted data. All four lines send a stop bit of 1, as well. CONTROLLER-SPECIFIC NOTES The AX211 is based on the 8051 architecture, which is an eight-bit processor core that is very common among embedded parts. The original 8051 ran at around 1 MIPS, but improvements in architecture and clock speed allows the 8051 present in the AX211 to run at 50 MIPS. The 8051 instruction set had a single undefined opcode of 0xa5, which is used by Appotech to implement 32-bit instruction set extensions. The hardware SD protocol engine only handles transferring bytes (OSI layers 1 & 2). The higher-level details of the SD protocol set is implemented entirely in software, making for a compelling attack surface. When the processor first starts up it attempts to load a valid flash image off of the NAND. While it is in this state, it can respond to a limited set of SD commands, including "RESET", "Set CRC7 required", and "Set voltage". It will not respond to the "Go Ready" command until it has loaded its firmware. There are four copies of the AX211 firmware, located at NAND offset 0x0000, 0x10000, 0x20000, and 0x30000. Each copy is protected with its own error correcting code. If all four copies are sufficiently damaged, or if the NAND is missing or blank, then the card will never fully boot, and will never be able to go ready. FACTORY PROGRAMMER As noted previously, we had found a copy of the Windows-based AX211 factory programming tool off of Chinese-language file sharing websites. The full AX211 factory programming suite is a combination of this software and a proprietary (and unknown to us) hardware tool. We suspect the hardware is based on the AppoTech AX2002 processor, which also uses an 8051-compatible instruction set. The device connects to the main PC through USB, and has the USB ID string of "appotechcksd". This allows the software to identify the programmer regardless of its product/vendor ID. We believe the device may have a small display, or else it prints debugging information out a serial port. The programming software is almost entirely in Chinese. It does not contain a codepage identifier, so a system must be set to run non-Unicode programs as "Chinese (PRC)", which selects an encoding of GB-18030. If you do not do this, the software will display gibberish in place of Chinese text. The programmer control software appears to be fully automated. Without the programmer hardware, we can only configure the software and see where, presumably, individual hardware programmers would appear in the user interface. Therefore, we must resort to disassembling the binary to determine how it works. Code flow analysis indicates that when the software initially detects the attachment of factory programming hardware via USB, it opens the device and sends a binary file called "2005FM.BIN". 2005FM.BIN is a raw opcode stream that is loaded into the programmer hardware's 8051 (not the SD target) at address 0. The program begins running from address 0, and begins doing some initialization. It then displays the string "RUN" through the putative logging interface on the programming hardware. After this, it awaits further control instructions from the host. The microcontroller used in the programmer hardware does not contain an SD/MMC host controller. This is fortunate for us, as it means the SD programming interface is implemented entirely using GPIOs. This makes it easy to trace all signal changes through static code analysis. When programming begins, the programmer places the SD card into a special mode where it will accept and execute programs. It does this by sending SD CMD0 (card reset) followed within 20 msec by CMD63 with the arguments 'A' 'P' 'P' 'O' and the appropriate crc7 checksum. The card then responds with the following sequence: {0x3f 0x00 0x9d 0x20 0x0b 0x35} | | | | | \-- CRC7 checksum, plus stop bit | \----\----\----\------- Unknown values \---------------------------- The command we sent (CMD63) The program to be executed on the SD card is then loaded to offset 0x2900, and will begin executing immediately from within an interrupt context. Therefore, the first thing the routine should do is escape the context by changing the return value, then issuing a "reti" command. When the host software detects a new card to be programmed, it informs the programmer to put the card into this special factory mode, and then begins uploading firmware and communicating with the card. The first program to get loaded is a small piece of test code called TestBoot.bin. The program performs a few sanity checks, then returns pass or fail by sending the result out the Command line. Once the program finishes, control returns to the AX211 ROM. After TestBoot.bin runs, the programmer loads a file called communication.bin. This program contains a full interrupt vector table, and is therefore able to replace functions such as SD communication and NAND access. It uses this to load a new, expanded set of communication options, including the ability to load and execute another program at memory offset 0x800. Because it has its own interrupt vector table, it loads at offset 0, and takes control of the entire card. Once the communications program is loaded, the programmer uses it to load more advanced programs that actually deal with the NAND flash component. Programs are loaded at offset 0x800 in order to allow the communications program (and its interrupt vector table) to stay resident. The first program loaded is FLASH_SCAN.BIN, which is responsible for scanning the flash to determine its properties. This can be useful when operators are not familiar with the type of NAND they're working with. This program is able to inform the host of properties such as the page size, flash size, and ECC data. The host can then use this information to construct a Card ID block. It's important to note that the host controller can choose to ignore the information coming from the flash, and can program invalid flash sizes. This can occur either deliberately or accidentally. Thus, some "counterfeit" cards may simply be a result of an operator forgetting to change the settings when programming a new batch of NAND. After the flash has been scanned, a program called FLASH_PRO.BIN is loaded. This is responsible for the actual programming of the NAND with the requisite firmware file. With FLASH_PRO.BIN running, the programmer must feed the card a firmware file. A wide variety of firmwares are bundled with the programming software, and vary between the vendor, page size, block size, number of planes, and the number of chip enables. The host picks an appropriate firmware file to send, and writes it out to the card. After programming is complete, the programmer resets the card and formats it. Depending on the size of the card, the programmer places either a FAT16 or a FAT32 header on the card, and creates the actual file allocation table. It is interesting to note that there are two DOS MBRs stored in the 2005FM.BIN programmer firmware, complete with an NTLDR boot sector. This gets placed at the head of the card, with the actual file allocation table following shortly after. The upshot of reversing the factory programmer binaries is the development of our own routines which is capable of knocking the AX211 and loading our own code onto the device. AX211 FIRMWARE REVERSING The contents of the programs run on the AX211 has also been analyzed. In addition to confirming the host-side initialization protocols documented above, the firmware reversing effort has revealed more details on the allocation of instruction set enhancements and the Function Specific Registers (FSRs). FSRs are memory-mapped registers that are used to control and configure the state of the hardware. For example, the pins required to turn NAND I/Os into GPIOs and toggle their values is contained in the FSR region; therefore discovering the FSR map is an important component of achieving maximum utility from the SD card controller. Opcode extensions on the 8051 are accomplished by using the escape opcode 0xA5. This is the only opcode function in the 8051 which doesn't have a pre-defined value. Typically, the escape opcode is followed by one or more bytes that specify which escape sequence to use. In our investigation of the firmware, we have only seen one and two-byte escape sequences. Two-byte sequences require that the lower nibble of the first escape opcode to be 1's. We have modified the 8051 processor module in IDA to handle these escape sequences gracefully. Our current theory is that the opcode extensions are used to trigger various NAND interface functions, such as transmitting a command cycle or computing ECC on a loaded page of memory. The FSR map was determined in part through static binary code analysis in IDA, and in part through a dynamic fuzzing infrastructure. For example, to determine which FSR was used as a GPIO, we wrote a small assembly stub that would set a pseudo-random (based upon a seed value, so we could reproduce experiments later on) selection of four FSR registers within the 128-byte window of valid FSRs to 0xFF, then 0x00, with a few microseconds' delay between. We had connected an oscilloscope to one of the NAND I/Os on the AX211, and monitored for changes in I/O status while running this code. We decided to write four bytes at once under the theory that we had to configure not only a data register, but also direction & configuration registers as well to be outputs for us to measure a change using the oscilloscope. With some luck, we were able to discover very quickly an FSR which, when written to, would cause some of the NAND I/O pins to flip on and off. Through this combination of static and dynamic analysis, we have confirmed the function of much of the FSR table, which looks like this: | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | | 8 | 9 | A | B | C | D | E | F | -----+-------+-------+-------+-------+-------+-------+-------+-------+ 80 | SDMOD | SP | DPL | DPH | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ 88 | SDOS | SDI4 | SDI3 | SDI2 | SDI1 | SDCMD | IACK | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ 90 | SDSM | | | SDBL | SDBH | | SDDL | SDDH | -----+-------+-------+-------+-------+-------+-------+-------+-------+ 98 | | | | | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ A0 | NTYPE | NCMD | NRAML | NRAMH | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ A8 | IE | NCMD1 | NCMD2 | NADD0 | NADD1 | NADD2 | NADD3 | NADD4 | -----+-------+-------+-------+-------+-------+-------+-------+-------+ B0 | | RAND | | | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ B8 | ER8 | | | | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ C0 | ER00 | ER01 | ER02 | ER03 | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ C8 | ER10 | ER11 | ER12 | ER13 | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ D0 | | | | | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ D8 | ER20 | ER21 | ER22 | ER23 | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ E0 | ACC | | | | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ E8 | | | NFMT | SDDIR | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ F0 | B | | NPRE1 | NPRE2 | | | PORT1 | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ F8 | ER30 | ER31 | ER32 | ER33 | | | | | -----+-------+-------+-------+-------+-------+-------+-------+-------+ Our presumption is many of the spots reading back as 00 are probably unused FSRs, so we have probably uncovered the function of most of the implemented FSRs. WORKING IT: A REPL DEBUGGER In investigating the actual runtime firmware of the AX211 card, it has become apparent that the entire SD engine at the protocol level is implemented in 8051 code. Only the SD physical layer (including both SPI and MMC mode) is handled in hardware. When the host controller sends a command such as CMD0, the host controller actually treats the command as an offset in a jump table. Application commands are also treated in a separate jump table. This means that it is a trivial matter to add an additional command. All one needs to do is co-opt one of the unused commands, which currently point to a default handler. There is plenty of space at the end of the firmware to add any additional commands. Now that the SD controller's functions are well known, we were able to implement a REPL-mode debugger firmware for the SD controller: root@bunnie-novena:~/ax211-code# ./ax211 -d debugger.bin FPGA hardware v1.26 Debug mode APPO response [6]: {0x3f 0x00 0xc1 0x04 0x17 0xab} Result of factory mode: 0 00000000 0f 41 1f 0f 0f 0f ff ff |.A......| Expected 0x00 0x00, got 0x0f 0x41 Loaded debugger Locating fixup hooks... Done AX211> help List of available commands: hello Make sure the card is there peek Read an area of memory poke Write to an area of memory jump Jump to an area of memory dumprom Dump all of ROM to a file memset Set a range of memory to a single value null Do nothing and return all zeroes disasm Disassemble an area of memory ram Manipulate internal RAM sfr Manipulate special function registers nand Operate on the NAND in some fashion extop Execute an extended opcode on the chip reset Reset the AX211 card help Print this help For more information on a specific command, type 'help [command]' AX211> As one can see, the features of the debugger include generic options such as memory manipulation (peek/poke), but also includes more powerful features such as disassembly, and chip-specific features such as NAND manipulation via the internal SFRs. DEBUGGER IMPLEMENTATION The debugger relies upon two key mechanisms that were reverse-engineered in the course of this study: (1) the ability to load small stubs of arbitrary code to bootstrap the system and (2) the ability to pass messages back and forth from the AX211 using the SD PHY layer. The footprint of the native 8051 code for the debugger is limited to 512 bytes; this is the stub size afforded by the mask ROM bootloader on the AX211. Therefore, in order to implement a rich command feature set, the debugger functions are partitioned between a host CPU (the ARM i.MX6) and the target 8051-based CPU. For example, disassembling a particular region of memory breaks down to a script executed on the host side that drives the native AX211 stub code to dump the requested portion of memory to disassemble, followed up with the disassembly algorithm running on the host ARM CPU. The only way to talk to the AX211 is via the SD interface. Therefore, the interactive REPL environment must run on the host ARM system, which is equipped with the requisite terminal interface features such as a ssh console. The REPL environment then translates the user's requests into bundles of SD commands. We take advantage of the command/response token protocol built into the SD-PHY spec (see page 8 of SD Card Physical Layer Simplified Specification v2.00 for more information). The command/response protocol runs bidirectionally on a single wire, the CMD line of the SD-PHY interface, and is timed by the CLK line. Command tokens are 48-bits long, and contain a CRC along with some static header bits. Response tokens are also 48 bits long, and similarly feature CRC protection along with different static header bits. The SD commands originating from the REPL environment are transmitted to the AX211 via the FPGA built into the host system. The FPGA presents a “gpio-like” API for the SD host PHY: one register for data output, one register for data input, and one register to bit-wise set the data direction. The SD commands are received on the AX211 and processed by the hardware SD engine attached to the embedded 8051 CPU. The state machine handles receiving the data and computing/checking the CRC. Once a complete packet is received by the state machine, the 8051 is notified of the packet's arrival via an interrupt. Therefore, the 512 bytes of native 8051 code for our debugger contains the following routines: * An entry sequence that resets the execution environment and redirects ongoing code execution into the debug.bin patch * An interrupt handler for receiving SD command packets * Code to set the ISR handler to our new interrupt handler * A jump table to parse incoming SD commands * Helper routines, such as the response transmission and wait loops * Implementation of all the basic stub commands: * hello - inform the host we're running * echo - loop back the request * peek - request contents of XRAM * poke - set contents of XRAM * jump - run code at an address * nand - manipulate the NAND hardware registers * SFR/IRAM get/set routines - more on this below * extended opcode - execute an extended opcode * error - return an error code CROSS-MEMORY AREA ACCESS The 8051 has two different kinds of memory, each of which is further subdivided: IRAM - located on-chip. 256 bytes in total. Accessed using "mov" instruction. The IRAM map is as follows: * IRAM address 0-7 corresponds to CPU registers R0-R7 * IRAM address 8-0x7f are on-chip RAM * IRAM address 0x80-0xff are Special Function Registers (SFRs). SFRs are locations that are mapped to hardware functions, such as the NAND command engine or GPIO XRAM - 16 kilobytes of "External" RAM (this memory was a physically separate chip in the original non-embedded 8051 implementation). Accessed by storing an address in DPTR (which is really made up of SFR 0x82 and 0x83, called DPL and DPH respectively) and either loading to the accumulator using "movx A, @DPTR" or storing from the accumulator using "movx @DPTR, A". The XRAM map is as follows: * 0x0000 - 0x0006 is reserved. Contains 0x51 0x00 0x00 0x00 ... * 0x0007 - 0x01ff is protected and returns 0xff. The CPU and NAND block can't read this range, but the SD block can. * 0x0200 - 0x0202 is interrupt vector 0 (SPI) * 0x0203 - 0x0205 is interrupt vector 1 (other SPI) * 0x0206 - 0x0208 is interrupt vector 2 (NAND) * 0x0209 - 0x020b is interrupt vector 3 (unknown) * 0x020c - 0x02af is general-purpose RAM * Code execution for APPO factory mode begins at offset 0x2900 * 0x2ba0 - 0x2bff contains something interesting, and I'm not sure what * 0x2c00 - 0x3fff is read-only and contains zeroes * 0x0000 - 0x3fff is mirrored at 0x4000, 0x8000, and 0xc000 The division of 8051 memory space into IRAM and XRAM presents a unique challenge for debugger implementation. Not being able to index into IRAM means that one cannot dereference a variable pointer into IRAM. In the 8051 ISA, only constant values can be dereferenced into IRAM. This pseudo code draws a loose analogy of the situation in C-like syntax: iram_vaule = *iram_pointer; // ERROR: variable values not allowed iram_value = *(0x42); // ACCEPTABLE: only hard-coded constant values allowed This limitation of the 8051 architecture present a challenge in implementing a key feature of the debugger, namely enabling the dynamic exploration of the IRAM and associated SFR space. Fortunately, on the AX211, the debugger code is loaded into XRAM, which can be indexed via the DPTR register using the MOVX instruction. This allows the debugger to manipulate its own code, thereby allowing us to change, for example, the offset constant to a MOV instruction inside the debugger. Therefore, in order to implement peek and poke in the IRAM space (and likewise enable convenient exploration of the SFRs), we reserve three-byte slots in debug.bin using a sentinel sequence of invalid opcodes that are unique and searchable. Once debug.bin is loaded, one of the first tasks the host program does is scan the loaded program for the unique sequences and records their offsets. These sequences are then replaced on-demand with the appropriate instructions to implement the required MOV opcodes to explore arbitrary locations in IRAM. Below is a concrete example of this technique as implemented in the debugger. The following code snippet is from debugger.asm: cmd_sfr_get: ; This will get replaced by "mov 0x20, [SFR]" at runtime .db 0xa5, 0x60, 0x61 mov 0x21, #0 mov 0x22, #0 mov 0x23, #0 sjmp xmit_response Here, we are looking at the “SFR get” stub. The arguments to return to the host are loaded into IRAM locations 0x20-0x23. For example, mov 0x21, #0 puts the constant value 0 into IRAM 0x21; so in this stub, locations 0x21-0x23 are set to 0, and 0x20 is intended to have the value of the queried SFR. A sequence of three bytes, “0xA5 0x60 0x61” is used as a placeholder for a different “mov” instruction that will be slotted in. 0xA5 is the sole invalid opcode in the 8051 instruction set, and therefore it is an “almost safe” opcode (bar collisions with other opcode extensions) to use as the starting marker for a sentinel sequence. The remaining bytes “0x60 0x61” are simply chosen to be unique and non-colliding with other opcode extensions. When the user requests the value of an SFR via the REPL interface on the host by typing, for example, sfr -r 0xA0 the host issues a command to poke the cached location of the sentinel sequence “0xA5 0x60 0x61” with the sequence “0x85 0x20 0xA0” which represents the instruction mov 0x20 0xA0 Recall that the only difference between IRAM and SFR is that SFRs have an address greater than 0x80; and furthermore all arguments to the mov are constant, therefore this is a valid 8051 instruction that implements the requested command via the REPL interface. Now that the correct instruction has been installed in the cmd_sfr_get routine, the actual command to run this callback is issued on the SD interface and the requested SFR is returned to the host. This enhanced ability to interrogate the SFR space dynamically allowed us to greatly expand our map of the special function registers. EXTENDED OPCODES A similar method was used to also explore the extended opcode space for the 8051. As noted in the previous section, “0xA5” is the sole illegal instruction in the native 8051 instruction set, and in enhancements such as the one found in the AX211, it is used as an escape sequence to specify an extended instruction set. As a (entertaining) side-note, the sparse literature available on the AX211 claims that the on-board 8051 is a 32-bit processor. We originally thought this was an amusing “lost in translation” moment, but in fact, the AX211 implements a set of instructions that operate on 32-bit data types using extended opcodes, thereby lending credibility to the “32-bit” label. SFR locations 0xC0-3, 0xC8-B, 0xD8-B, and 0xF8-B were identified as functioning 32-bit registers, and opcodes were discovered that could reverse the order of bits in these registers, invert the contents of these registers, and clear the contents of these registers to zero. There may be other opcodes present as well; code has been found implementing CRC16 checksums using these 32-bit enhancements, and most likely these enhancements also enable the implementation of MLC/TLC scrambling algorithms. The BCH ECC computations, however, are handled by a dedicated hardware coprocessor. SUMMARY An approach has been disclosed for the exploration and exploitation of the embedded controller found within a particular type of SD card. The approach consists of a combination of static code analysis, and dynamic fuzzing analysis. A "secret knock" for uploading code into the controller was found, and through this mechanism we explored the register map and extended opcodes of the microcontroller. Significantly, in this particular device all of the SD protocol-layer commands are implemented in software. This allowed us to redefine the SD protocol set and implement, as a demonstration, a REPL-mode debugger for the SD card.

Other people: Xobs