In a previous post [1] I commented on how to port the Micropython Interpreter to the Rasperry Pi. There I went over the basics of porting Micropython to a new hardware and showed some ways on how to configure your port. This post builds upon that foundation and uses the Micropython which did run in principle, but ran into some nasty problems when executing code. Often I would receive failed assertions which could not fail.

At first I (with the help of my colleague) tested basic statements of Python and watched the Raspberry Pi fail. Basic print-Statements with a fixed string, basic variable operations, later if, while and for. We did compile Micropython to talk to use really verbosely. It should output every last detail of what it does.

What changed from last time

But first I need to talk about what changed since the first post. I removed the neccessety to use newlib as external library and started using the libC-Implementation of Micropython. This implementation is not complete but complete enough to run the interpreter on a new hardware. Functions like fopen do not exist, but we don't need them anyways (until now).

Also I build a new build-system. I incorporated description-files into my PiOS-project, which describe the specifications of the hardware to be built for. In the Makefile of the Raspberry Pi-port there is a variable BOARD which can be set to rpi or qemu. Yes, the same code can be compiled for QEMU, which is pretty great for testing and finding differences.

Making Micropython talk to us

There are two main ways of making Micropython talk a lot. The first one is settings the flag mp_verbose_flag at runtime to something greater than 2. I believe this flag will not be used in future (or something), it is used only once in the entirety of the code. This flag decides whether Micropython will print the Bytecode and a string-representation of the Bytecode or not.

Then there are DEBUG_printf-makros inside the code. For example like this (from bc.c [2]):

#if 0 // print debugging info
#define DEBUG_PRINT (1)
#else // don't print debugging info
#define DEBUG_PRINT (0)
#define DEBUG_printf(...) (void)0
#endif

This code disables by default loads of output inside the code-file. Many code-files of Micropython have statements like these on the top. I changed it per file to [3]:

#if MICROPY_DEBUG_VERBOSE // print debugging info
#define DEBUG_PRINT (1)
#else // don't print debugging info
#define DEBUG_PRINT (0)
#define DEBUG_printf(...) (void)0
#endif

Then I can define the preprocessor constant MICROPY_DEBUG_VERBOSE in the ports mpconfigport.h-file. This will enable output of every last part of Micropython, i.e. also the Garbage-Collector.

What is wrong?

So I tried several statements all seemed to work fine, except for, while and if-statements. When looking through the Bytecode I found JUMP-statements with wrong Labels, like for the following code:

do_str("for i in range(10):\n    print(i)", MP_PARSE_FILE_INPUT);

I got the following Bytecode and String-Representation:

File <stdin>, code block '<module>' (descriptor: 66fb0, bytecode @67040 41 bytes)
Raw bytecode (code_info_size=6, bytecode_size=35):
 03 00 00 00 00 00 06 09 10 29 00 00 ff 80 35 0f
 80 30 24 82 25 1c 81 6f 1c 82 25 64 01 32 81 e9
 30 8a f0 36 eb 7f 32 11 5b
arg names:
(N_STATE 3)
(N_EXC_STACK 0)
  bc=-1 line=1
  bc=8 line=2
00 LOAD_CONST_SMALL_INT 0
01 JUMP 4294938425
04 DUP_TOP
05 STORE_NAME i
08 LOAD_NAME print
11 LOAD_NAME i
14 CALL_FUNCTION n=1 nkw=0
16 POP_TOP
17 LOAD_CONST_SMALL_INT 1
18 BINARY_OP 18 
19 DUP_TOP
20 LOAD_CONST_SMALL_INT 10
21 BINARY_OP 25 __lt__
22 POP_JUMP_IF_TRUE 4
25 POP_TOP
26 LOAD_CONST_NONE
27 RETURN_VALUE

For comparison this is the QEMU-output:

File <stdin>, code block '<module>' (descriptor: 6efa0, bytecode @6f030 41 bytes)
Raw bytecode (code_info_size=6, bytecode_size=35):
 03 00 00 00 00 00 06 09 10 29 00 00 ff 80 35 0f
 80 30 24 82 25 1c 81 6f 1c 82 25 64 01 32 81 e9
 30 8a f0 36 eb 7f 32 11 5b
arg names:
(N_STATE 3)
(N_EXC_STACK 0)
  bc=-1 line=1
  bc=8 line=2
00 LOAD_CONST_SMALL_INT 0
01 JUMP 19
04 DUP_TOP
05 STORE_NAME i
08 LOAD_NAME print
11 LOAD_NAME i
14 CALL_FUNCTION n=1 nkw=0
16 POP_TOP
17 LOAD_CONST_SMALL_INT 1
18 BINARY_OP 18 
19 DUP_TOP
20 LOAD_CONST_SMALL_INT 10
21 BINARY_OP 25 __lt__
22 POP_JUMP_IF_TRUE 4
25 POP_TOP
26 LOAD_CONST_NONE
27 RETURN_VALUE

This was the moment I went to the Github-Issue-Tracker of Micropython and looked for help [4]. As Mr. George pointed out the errors might be due to the inability of the ARM1176jzf-s processor to load unaligned data. Apparently the ARMv5 processors where completely unable to load unaligned data, that is data which is not aligned with the word-size of the processor, i.e. with 32-bit words aligned to 4-byte addresses (the address does not end with 0b00). In ARMv6 ARM added an extention which allowed loading and storing unaligned data. The ARM1176jzf-s specifically has a system coprocessor which could be set to allow unaligned accesses [5].

Setting the U-flag (22nd Bit) c1-control register of the C15 coprocessor allows us to access unaligned data without the processor throwing an exception to the kernel to resolve the issue. So this is the code, which I added to my bootup-assembly with the help of this RaspberryPi-Forum post [7].

/// load C15 control register c1-c1,0,0 (c1 Control Register) into r0
mrc p15, 0, r0, c1, c0, 0
ldr r1, =0x00400000     /// set bit 22 (U-bit)
orr r0, r1
mcr p15, 0, r0, c1, c0, 0

Further Reading

I found the article under [8] especially helpful in understanding the problem and finding some solutions.

Possible solutions which I tried before, was to resolve the issue with the wrongly accessed JUMP-label by using a function, which the compiler cannot optimize (__attribute__((optimize('0')))) so the two Byte-loads cannot be optimized to a single 16-bit load. This worked well for outputting the Bytecode as strings, but the code crashed before execution due to another unaligned (I guess). The assertion *rc->kind == MP_BYTECODE * failed, which prevented Micropython from executing the code.

References

[1] https://www.stefannaumann.de/en/2017/05/porting-micropython-to-raspberry-pi/
[2] https://github.com/micropython/micropython/blob/master/py/bc.c#L38-L43
[3] https://github.com/tuc-osg/micropython/blob/master/py/bc.c#L38-L43
[4] https://github.com/micropython/micropython/issues/3201
[5] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0301h/ch03s02s01.html
[6] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0301h/Bgbciiaf.html
[7] https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=126891
[8] http://antirez.com/news/111