Arduino without the IDE – An intro to UNIX Make

Recently I’ve been having a go at make. Make is an ancient and powerful UNIX utility that we can use for automating a software build process. But why do we need this when the Arduino IDE does this for us? For me, this comes down to the following:

  1. I wanted to have full control over the build process and all files that are included such as the Arduino cores; as the lifetester project nears maturity, I want to be in control of all files included and and what they contain.
  2. I didn’t like the way that all tabs in the Arduino sketch are stitched together (see Arduino build process). This means that any global variables that you declare as static within a module are then brought into the same file and are no longer private.
  3. Lastly, I like to use Sublime Text. I love the text highlighting and keyboard shortcuts. It really speeds up editing for me. Since discovering it, it’s been hard for me to accept anything else including the Arduino IDE.

I should say at this point that there is already a well developed makefile for the Arduino project here.  For me, it was impenetrable and so I went through the exercise of writing my own to get some idea how this mysterious tool works.If this interests you, then read on. Otherwise go to the link and check out a copy.

So what I was looking for a way to write C files and build them into a binary that I could upload onto the Atmega328 without the IDE. Compiling and linking C files is in UNIX (or Windows) is straightforward – just invoke the C compiler with cc. What does the Arduino IDE do then? All you have to do is turn on verbose in settings and you can see the commands issued in the console (and loads of other output too) at the bottom of the IDE. To demonstrate this, I saved a copy of Blink.ino as MyBlinkTest.cpp and copied the relevant files from the Arduino cores (/usr/share/arduino/hardware) and variants folders to the working directory. I called the following commands in the prompt…

$ avr-gcc -Os -DF_CPU=16000000UL -mmcu=atmega328p -c MyBlinkTest.cpp main.cpp new.cpp Stream.cpp wiring.c wiring_digital.c WString.cpp
$ avr-gcc -mmcu=atmega328p main.o MyBlinkTest.o new.o Stream.o wiring.o wiring_digital.o WString.o -o MyBlinkTest.elf
$ avr-objcopy -O ihex -R .eeprom MyBlinkTest.elf MyBlinkTest.hex
$ avrdude -F -V -c arduino -p ATMEGA328P -P /dev/ttyACM0 -b 115200 -U flash:w:MyBlinkTest.hex

…but it didn’t work. I found that I also needed to add “Arduino.h” to the top of MyBlinkTest.cpp. This is because the Arduino-specific commands such as digitalWrite etc. need to be defined and this header points to the functions where that happens. There were then a few places in the Arduino core code that I had to replace  instances of <Header.h> with "Header.h". The angle braces were telling the compiler to search system directories rather than the working directory.  Then it worked!

Clearly, this approach is limited. We would have to write out all the c files and object files explicitly, and copy files from the core directory and then remember all the commands. Not really feasible and very messy especially as the project starts to grow.

Enter make…  What it does is automate these commands by manipulating text and punching it into the command line for us. You can see the process in the commands above which I’m going to breakdown next and implement them in a makefile. Before I go on, it’s worth going over the basics of make here. A makefile consists of a number of ‘rules’ that each create a ‘target’ based on ‘prerequisites’ (dependencies) and commands. They look something like this:

target [target ...]: [component ...]
   Tab ↹[command 1]
           .
           .
           .
   Tab ↹[command n]

In make, variables are called ‘macros’ and we write them as…

MACRO=definition

and use them like this…

${MACRO}

This just means that whatever text we defined for that particular macro will be inserted where we specify.  So our hello world application would look like this…

MACRO1=hello
MACRO2=world!

test:
	@echo ${MACRO1} ${MACRO2}

Armed with this basic knowledge, let’s go through a makefile for an Arduino build – the classic blinking LED. You’ll find a copy of these files here if you’re interested in having a starting point for your own adventures with make.

Step 1: Compilation

First, we call the avr-gcc compiler, with a list of c and cpp files that we want to compile into object (.o) files. But in the makefile however,  you’ll see that I haven’t specified a file list however. I’ve said that the target will be a group of object files based on all .c and .cpp files in the DEPS macro combined with MyBlinkTest.cpp (blink.ino renamed). Using the VPATH macro, I’ve pointed the compiler to the Arduino core and variants directories. This is where all the important under-the-hood stuff is stored for building Arduino sketches. Note the additional -I flag in the call to avr-gcc which tells the compiler where to look for header files.  You’ll also see that I’ve sent all the object files to a build directory that is created if it doesn’t exist. Last but not least, there’s an important make idiom in the compiler call in the use of $^  which is an internal macro or automatic variable which expands to a space delimited string of prerequisites (‘implicit’ source): a list of all our .c and .cpp files.

VPATH=/usr/share/arduino/hardware/arduino/cores/arduino
VARIANTS=/usr/share/arduino/hardware/arduino/variants/standard
DEPS=${VPATH}/*.c ${VPATH}/*.cpp MyBlinkTest.cpp
BUILD_DIR=Build
CC=avr-gcc
MMCU=-mmcu=atmega328p
CFLAGS=-Os -DF_CPU=16000000UL ${MMCU}

${BUILD_DIR}/*.o: ${DEPS}
	mkdir -p Build/
	${CC} ${CFLAGS} -c $^ -I ${VARIANTS} -I ${VPATH}
	mv *.o ${BUILD_DIR}/

Step 2: Linking

The linking step simply takes all the object files that we’ve generated and bundles them together in one .elf file. Again, I’ve used the implicit source variable ($^) and the target variable $@  which substitutes in the name of the target which in this case is ${PROGRAM}.elf  and evaluates to MyBlinkTest.elf.

PROGRAM=MyBlinkTest

${PROGRAM}.elf: ${BUILD_DIR}/*.o
	${CC} ${MMCU} $^ -o ${BUILD_DIR}/$@

Step 3:  File conversion

OBJCOPY=avr-objcopy

${PROGRAM}.hex: ${BUILD_DIR}/${PROGRAM}.elf
	${OBJCOPY} -O ihex -R .eeprom $&lt; ${BUILD_DIR}/$@

Using the avr-objcopy command the .elf file is converted into a standardised .hex format. $<  is used here as it stands for the first prerequisite. There is only one so you get the idea.

Step 4: Upload

PORT=/dev/ttyACM0

upload: ${BUILD_DIR}/${PROGRAM}.hex
	avrdude -F -V -c arduino -p ATMEGA328P -P ${PORT} -b 115200 -U flash:w:${BUILD_DIR}/$&lt;

Heavy lifting done. Now it’s time to upload our beautiful code onto the Arduino with this last command which calls avrdude. Note that I’ve used $<  again to substitute in the name of the prerequisite and a macro to hold the name of the port.

Other things to note

  • Tabs are tabs in make! Don’t indent by four spaces and expect that to be equivalent. Make only understands tabs. You have been warned.
  • It’s worth defining the first rule (default target) as the one where all the important targets are specified. I’ve called this all.
  • You can also selectively just compile or upload only if you have defined rules for this by simply calling make compile or make upload from the command line respectively.
  • A clean rule is also a good idea. I’ve defined one here which just deletes everything in the build directory.
all: ${BUILD_DIR}/*.o ${PROGRAM}.elf ${PROGRAM}.hex upload

# option to compile only without upload/install
compile: ${BUILD_DIR}/*.o ${PROGRAM}.elf ${PROGRAM}.hex

upload: ${BUILD_DIR}/${PROGRAM}.hex
avrdude -F -V -c arduino -p ATMEGA328P -P ${PORT} -b 115200 -U flash:w:${BUILD_DIR}/$&lt;

clean:
rm -f ${BUILD_DIR}/*

New job!

Sorry it’s been so long! After finishing my contract at Sheffield University, I was out of work for a couple of months and desperate to get a job sorted. To this end, I’ve been working hard to improve my knowledge of C and it seems to have worked! I was offered a job as a graduate embedded software engineer at Cambridge Medical Robotics three weeks ago. So far so good – They’re a really friendly bunch of talented people and I’m learning a lot!

A big thanks to my mate Jan for putting me forward for the job and inspiring me to go for it and thanks to Al Kelley and Ira Pohl for their book on C. If you’re thinking about a move into software, then my advice would be to get stuck in. My experience was really positive at all the interviews I attended and hard work is rewarded. Put in the time, really learn your stuff and it will happen.

Welcome to my blog

Thanks for visiting my blog. I’m keeping a record of my work on hobby projects relating to my personal interests that cover instrumentation, embedded software, electronics and general tinkering around with stuff. I am passionate about technology and making stuff work but I really enjoy learning and playing with things. This is my chance to give back and share what I’ve learned from helpful online resources and people that have contributed to my knowledge and development. I hope you like it!

Dave