Show make.asc syntax highlighted
EasyBuild Reference
===================
The EmStar Team <emstar-design at cens dot ucla dot edu>
$Id: make.asc,v 1.25 2004/11/19 04:25:23 jelson Exp $
The EmStar build system, 'easybuild', ensures that when you type
"`make`", all targets will build correctly. EasyBuild generates a
makefile that is guaranteed to have correct dependencies, such that
when any subset of the source or header files change, libraries and
executables are properly rebuilt in the correct order.
EasyBuild is specifically designed to handle cross-compilation and
concurrent compilation for multiple targets, as this is the common
case with embedded systems development. It can also compile certain
targets differently or exclusively for certain platforms. The
specifications for the build are encoded in special `BUILD` files that
describe what libraries and daemons to build.
Using EasyBuild to Build EmStar
-------------------------------
In the common case, building EmStar is as easy as typing `make`:
$ make
That's it! By default, 'easybuild' will build binaries for the
`i686-linux` target, and place all the generated binaries in a
directory called `obj.i686-linux`.
To build EmStar for a different platform, give `make` an `ARCH`
argument. For example, to build binaries for the Stargate platform,
type
$ make ARCH=stargate
This places generated files in a directory called `obj.stargate`.
If you're planning on building for Stargate all day, you can also set the
`ARCH` environment variable, then use `make` without arguments:
$ export ARCH=stargate
$ make
*If you encounter problems*:
- If you're using CVS rather than a release, be sure that you have
checked out the entire repository using `cvs update -d`.
The `-d` argument ensures that completely new directories are also
checked out.
- Moving files around or deleting files sometimes leads to
changes to dependencies that confuses EasyBuild. Incorrectly-built
binaries will not occur; at worst, the build itself will fail with
error messages about missing files.
In these cases, try deleting `obj.<platform>`. In some cases
you will only need to delete some of the files from the
`obj.<platform>/.build` directory.
Binaries Generated by EasyBuild
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
EasyBuild stores all targets in a per-platform directory hierarchy
that mirrors the hierarchy of source tree. Specifically, it mirrors
the hierarchy of the files called `BUILD` in the source tree that
control the build. Each target built from a particular `BUILD` file
will be stored as the target name relative to the path of the `BUILD`
file that generated it. For example, the target `libmisc.a` built by
`libmisc/BUILD` for platform `stargate` will be stored in
`obj.stargate/libmisc/libmisc.a`. All object files and dependency
files related to each target are stored in a directory hierarchy under
`obj.<platform>/.build`.
Partial Builds and Other Build Commands
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The default `make` target is to do a complete build. However, you can
also specify a specific file to build by specifying its full path in
the `obj.<platform>` directory. For example, to only build the
`fusdd` executable, as well as all of its dependencies:
make obj.i686-linux/fusd/fusdd
The other supported targets are `make clean` to do a fresh build and
`make test` to build and execute the regression test suite.
Creating new targets with BUILD files
-------------------------------------
In this section, we will describe how developers can add new
targets to the BUILD files as new software is written.
EasyBuild parses a set of configuration files call `BUILD` files, to
generate a single, unified Makefile. The advantage of this approach
is that dependencies across the entire tree can be correctly computed.
The `BUILD` file in the root of the EmStar distribution is the file
that EasyBuild starts with. This file in turn includes other `BUILD`
files from other parts of the tree, and thus constructs the complete
build.
Basic EasyBuild Syntax
~~~~~~~~~~~~~~~~~~~~~~
EasyBuild parses a syntax that include primarily C-like brace blocks
and comma-separated lists. Comments can be included using the `#`
mark, just as in a Makefile. In most cases, whitespace is not
important. The only exception to this are certain directives that are
copied verbatim into the Makefile, and are terminated by end of line.
Top-Level EasyBuild Config Blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are four kinds of top-level blocks: `include`, `build`,
`test`, and `literal`.
Include Blocks
^^^^^^^^^^^^^^
The `include` block contains a comma-separated list of directories.
EasyBuild will search each listed directory, and process the `BUILD`
file found there. Those `BUILD` files may in turn contain `include`
blocks. An example is shown below:
------------
include {
fusd,
audio_server,
emrun
}
------------
This means that after processing of the current BUILD file is
complete, 'easybuild' will process fusd/BUILD, audio_server/BUILD, and
emrun/BUILD.
Build Blocks
^^^^^^^^^^^^
The `build` block is the most commonly used block. This block
specifies targets to build and how to build them. Within the build
block, you can specify targets and many other options to configure how
the targets are built. These option settings are local to the `build`
block; that is, other build blocks will not be affected.
When you specify a build block, you first specify whether you are
building EmStar executable targets (`build bins`),
TinyOS/EmTOS targets (`build tos`), library targets (`build libs`),
or single object files such as kernel modules (`build objs`). The
special build type `build copy` simply copies files from the source
tree to the object-file tree without any compilation.
In addition, you may optionally specify a specific platform to build
for. This is useful for specifying targets that only build for
certain plaforms, e.g. a visualization GUI that does not build for
embedded platforms. More generally, this facility can customize the
build in arbitrary ways on a per-platform basis. The syntax enabling
builds for specific platforms is `build [bins | libs] for <platform>
[, <platform>]`. For `tos` targets, you may also specify `mote` or
`emstar` as "wildcard" platforms that will build for all Mote
platforms or all Microserver platforms respectively. Some examples of
the declaration syntax are shown below.
------------
build bins { }
build tos { }
build libs { }
build objs { }
build copy { }
build bins for i686-linux { }
build tos for mote { }
build tos for stargate, mica2 { }
------------
The syntax of the contents of the `build` block is described in
more detail in <<build_details,this section>>.
Test Blocks
^^^^^^^^^^^
The `test` block configures the behavior of the `make test` target
that runs the EmStar regression test suite. The `test` block contains
a comma-separated list of commands to run. Each command performs some
test, and exits with code 0 to mean "test passed", or code non-zero to
mean "test failed".
---------
test {
"opentest /dev/fusd-opentest",
ioctl-test,
}
---------
Literal Blocks
^^^^^^^^^^^^^^
The `literal` block is a useful escape hatch for solving problems that
are not easily addressed by the rest of the EasyBuild syntax. The
text contained in a `literal` block is copied verbatim into the
auto-generated Makefile. This can be used to insert additional rules,
add conditionals, and to set environment variables. Clearly,
`literal` must be used with caution and with an understanding of how
the EasyBuild system works and generates Makefiles. If you find
yourself using `literal` a lot to get around a missing feature of
EasyBuild, it may be worth considering the possibility of adding the
desired features.
Build Blocks [[build_details]]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The `build` block is the workhorse of the EasyBuild system. These
blocks can contain many different configuration settings, and can each
contain multiple targets to build. In this section, we will present
the options and format. Note that some options apply to all `build`
blocks, while others are only relevant to `build bins`.
Library Specifiers (bins only)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The library specifier options are the most commonly used. These
options list a set of libraries that will be included in the link
phase of all targets in the `build` block. There are two types of
library specifier: `local_libs` and `system_libs`. Both library
specifiers contain a comma-separated list of libraries within braces.
The library names are assumed to be prefixed by "`lib`".
An example of each kind of library specifier is shown below, along
with what it turns into on the linker command line. Note that while
in both cases the prefix `lib` is omitted from the input list, it is
resolved in different ways. In the case of `local_libs`, the trailing
pathname component will be converted to `lib<name>.a`.
---------
system_libs { m, jpeg }
# resolves to: -lm -ljpeg
local_libs { libmisc/misc, fusd/fusd }
# resolves to:
# obj.<platform>/libmisc/libmisc.a obj.<platform>/fusd/libfusd.a
---------
.Note
*****
As is usual with the C linker (`ld`), the order in which you specify
static library archives (`.a` files) matters. You must be sure to
list them in dependency order, such that all libraries that are
referenced *by* a particular library X, follow library X in the list.
If you have a circular dependency, you may need to list a library
twice to resolve all the references. Note that all `system_libs` are
listed before the `local_libs`.
*****
Flags Specifiers
^^^^^^^^^^^^^^^^
Flags specifiers allow you to add arbitrary strings into Makefile
variables that are included in the compilation command lines.
EasyBuild allows you to set four different flags: `cflags`,
`cxxflags`, `ldflags`, `tosflags`. When these variables are set, the
remainder of the line will be copied literally as the value of that
variable -- EasyBuild will not itself parse the value. The value will
be copied literally into the Makefile, which will then do its normal
processing, e.g. variable substitution, backtick processing, etc.
Examples of setting these variables are shown below:
---------
cflags := `pkg-config --cflags gtk+-2.0 gdk-2.0 atk`
cxxflags := -DMY_DEFINE_FLAG $(SOME_VARIABLE)
ldflags := --strip-discarded
---------
In the first example, the backticks will cause `make` to execute the
specified command in a subshell and then process the output into the
Makefile. The `cflags` value will be included in `gcc` command lines
when compiling C language targets.
In the second example, the `$(SOME_VARIABLE)` will be substituted by
`make`, according to whatever variables are defined at that point.
Makefile variables are defined in `make/make.platform`. The
`cxxflags` value will be included in `g++` command lines, when
compiling C++ language targets.
In the third example, the value of `ldflags` will be included in the
link step.
There is an additional flags variable, `tosflags`, that is used
only in `tos` build blocks. The `tosflags` variable can contain a set of
environment variables that will be set prior to running the TOS make.
.EasyBuild Makefile Variables
******
The following variables are defined in EasyBuild:
.`KCFLAGS`:
CFlags for compiling kernel modules
.`ARCH`:
The platform type.
******
Target Specifiers
^^^^^^^^^^^^^^^^^^
The `target` specifier lists a set of source files that will be
compiled and combined together to form an executable, library, or
object file. (`build copy` blocks are the exception; they are
required to contain exactly one source file.)
Here's an example of typical targets in a `build bins` block:
-------
build bins {
target Target1 {
target1/source1.c,
target1/source2.c,
common/common_code.c
}
target Target2 {
target2/file1.c,
target2/file2.c,
common/common_code.c
}
}
-------
In this example, two binaries called `Target1` and `Target2` will be
generated. (The executables will be placed in the obj.<platform>
directory, relative to the path of the BUILD file being parsed.)
Target1 will be built by compiling three source files, then linking
them together: `target1/source1.c`, `target1/source2.c`, and
`common/common_code.c`. Target2, similarly, will use the files listed
in its target block.
Note that Target1 and Target2 both use `common/common_code.c`.
EasyBuild is smart enough to only compile that source file once, and
link it into both executables. In fact, EasyBuild is even smarter: if
the same source file is used in two different build blocks, but those
build blocks have different compiler flags, EasyBuild will compile
`common_code.c` twice. Two object files will be generated,
named `common_code.o` and `common_code-2.o`, each compiled with
different flags. This makes it easy to have #ifdefs in source code
that change its behavior depending on which target will be using it.
There is also a directive called `simple_targets` which can be used to
compile targets that have only a single source file. For example, the
following two build blocks are identical:
------
# This is the simple_targets style
build bins {
simple_targets {
examples/example1.c,
examples/example2.c,
test/test1.c
}
}
# ...which is equivalent to the build block below
build bins {
target example1 { examples/example1.c }
target example2 { examples/example2.c }
target test1.c { test/test1.c }
}
-----
EasyBuild will act intelligently depending on the file suffixes:
```~~~~
[format="dsv",separator="|"]
File Suffix|Compiler|Linker
~~~~~~~
.c|gcc|if no C++, will link using gcc
.y|bison, then gcc|
.l|flex, then gcc|
.cc or .C|g++|will link using g++
~~~~~~~
Once all the sources are compiled, different things happen depending
on the type of `build` block. If it's a `bins` block, the files will
be linked together with the specified libraries to form an executable.
It is's a `libs` block, the files will be passed to `ar`. If it's an
`objs` block, `ld -i -o` will be called to link all the underlying
object files together into one big object file.
Conditional Compilation
^^^^^^^^^^^^^^^^^^^^^^^
Using EasyBuild, it's possible to specify that some targets should be
built only if certain environment variables are set, or not set.
These variables are usually set in the top-level file Make.conf, or in
your own enviornment.
Conditionals are specified using the "IF" clause to a build block,
like this:
------
# These binaries are only built if the BUILD_FOOBAR_APP environment
# variable is set to 1.
build bins if BUILD_FOOBAR_APP {
target foobar1 { foobar/example1.c }
target foobar2 { foobar/example2.c }
}
-----
You can specify multiple IF statements for a single build block, in
which case the results are logically-anded together:
------
# These binaries are only built if the BUILD_ALL_EXAMPLES *and* the
# BUILD_FOOBAR_APP environment variables are both set to 1.
build bins if BUILD_ALL_EXAMPLES if BUILD_FOOBAR_APP {
target foobar1 { foobar/example1.c }
target foobar2 { foobar/example2.c }
}
-----
Finally, logical negation is possible using the "!" character, just
like in C. For example:
-----
# These binaries are only built if IN_CYGWIN is *not* set to 1.
build bins if !IN_CYGWIN {
target something_that_only_works_under_linux { linux/example1.c }
target something_that_only_works_under_linux2 { linux/example2.c }
}
# These binaries are only built if the BUILD_FOOBAR_APP variable is
# set to 1, and the FOOBAR_APP_IS_BROKEN variable is *not* set to 1.
build bins if BUILD_FOOBAR_APP if !FOOBAR_APP_IS_BROKEN {
target foobar1 { foobar/example1.c }
target foobar2 { foobar/example2.c }
}
-----
There is one special environment variable understood by the build
system: USE_SHARED. This variable, typically set in Make.conf,
controls whether or not both shared (.so) and unshared (.a) libraries
are built for "build lib" blocks.
Building TinyOS Targets
^^^^^^^^^^^^^^^^^^^^^^^
EasyBuild supports integration of TinyOS targets into the EmStar build
system. This mechanism supports building both for Mote platforms such
as Mica2 and EmStar targets using the EmTOS wrapper library.
To add a TinyOS target to the build, add a `build tos` block to
the `BUILD` file. A `build tos` block can specify `cflags` and `tosflags`
that pertain to this build, and then specify one or more targets.
Each target specifies a target filename for the executable, and
inside the braces specifies the directory in which to run the `make`
to build the target.
When TinyOS targets are specified, they will build both for Mote and
EmStar platforms. That is, `make ARCH=mica2` will build all TinyOS
targets for Mica2.
The following block is an example of a `build tos` block. This block
will build two targets, the `EssDsp_smac` target and the `EssSink_smac`
target. `EssDsp_smac` will be built by entering the `tos-contrib/dsp/apps/EssDsp`
directory and running make, with the `USE_SMAC` environment variable set to 1
and the `-DHOST_MOTE_DATA_LENGTH=50` define added to the cflags.
------
build tos {
tosflags := USE_SMAC=1
cflags := -DHOST_MOTE_DATA_LENGTH=50
target EssDsp_smac { tos-contrib/dsp/apps/EssDsp }
target EssSink_smac { tos-contrib/tinydiff2/apps/EssSink }
}
------
.Important Note About TinyOS Builds
******
The TinyOS build system does not have any support for computation of
dependencies. This means that the EmStar build system has no way to
discover whether a particular TinyOS application needs to be rebuilt.
To address this, EmStar never rebuilds TinyOS targets if there is
already an executable present there. If you know that you made
changes that require rebuilding, you must force it to rebuild
by deleting the old executable and running `make` again. For
convenience, there is a script in `emstar/bin` called `tosremake`
which will do this for you.
For example, the following command will rebuild the EmTOS executable
called `obj.i686-linux/devel/ess/EssDsp_smac`. You can specify more
than one executable on the command line, although it will only
rebuild for the architecture of the first executable.
---
bin/tosremake obj.i686-linux/devel/ess/EssDsp_smac
---
There is also a script called `bin/tosinstall` that will use `uisp` to
program a Mote with an srec from the `obj.mica2` directory.
******
A Complete Example
~~~~~~~~~~~~~~~~~~
Just to pull all these features together, here is a complete example
of the EasyBuild syntax. To create this, we cut and pasted parts of
lots of existing BUILD files, to demonstrate the many different
features.
--------
# FrankenBUILD
# Build for x86 only and custom CFLAGS
build bins for i686-linux {
local_libs { emproxy/proxy, link/link, emrun/emrun,
libdev/dev, libmisc/misc, fusd/fusd }
system_libs { m }
cflags := `pkg-config --cflags gtk+-2.0 gdk-2.0 atk`
ldflags := `pkg-config --libs gtk+-2.0 gdk-2.0 atk`
target emview {
core/emview_node.c,
core/emview_slots.c,
core/emview_source.c,
# c++ code
cpp_test/cpp_from_c.cc,
# yacc code
emsim/emsim.y,
# lex code
emrun/emrun.l
}
}
# Build for all platforms
build bins {
local_libs { emrun/emrun, libdev/dev, libmisc/misc, fusd/fusd }
target binary-read { examples/binary-read.c }
target console-read { examples/console-read.c }
}
# Kernel modules
build objs {
cflags := $(KCFLAGS) -D__KERNEL__ -DMODULE
target kfusd.o { kfusd/kfusd.c }
target zero.o { test/zero.c }
}
# Libraries
build libs {
target libfusd.a { libfusd/libfusd.c }
}
# Regression Tests
test {
"opentest /dev/fusd-opentest",
ioctl-test,
"statetest 30",
scripts/deptest-regression.sh
}
# Configuration files that should be copied from the source tree
build copy {
target my_config.conf { configs/config-1.conf }
}
--------
Troubleshooting EasyBuild
-------------------------
I'm getting an `unresolved reference` error. How do I fix it?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First, examine the error output. Make a note of what symbol it was,
and where the symbol was referenced.
-------
$ make
/usr/bin/gcc -o obj.i686-linux/skeletons/motor_controller
obj.i686-linux/.build/skeletons/motor_controller/motor_controller.o
obj.i686-linux/libdev/libdev.a obj.i686-linux/libmisc/libmisc.a
obj.i686-linux/fusd/libfusd.a -L/usr/lib -Lglib/obj.i686-linux
-lglib-2.0
obj.i686-linux/.build/skeletons/motor_controller/motor_controller.o(.text+0x6e6):
In function `main': skeletons/motor_controller/motor_controller.c:501:
undefined reference to `emrun_init'
collect2: ld returned 1 exit status
make: *** [obj.i686-linux/skeletons/motor_controller] Error 1
-------
In the example above, the undefined symbol is `emrun_init`, and the
referring function was `main()` in `motor_controller.c`. In this
case, `motor_controller.c` is our own application code. This error
means we called a function from a library but did not include the
library in the link step.
Based on the name of the missing symbol (`emrun_init`) we might deduce
that it is part of the `emrun` library. If you can't figure out which
library to include, try using
http://cvs.cens.ucla.edu/lxr/source/emstar/[LXR] to search for the
symbol. When you find the library, include it in the `local_libs`
directive and try running `make` again. In this case, we needed to
add `emrun/emrun` to the `local_libs` directive.
Where is this function called? *My* code doesn't call it.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Libraries often depend on other libraries. So, when you include
certain libraries, you must also include other dependent libraries as
well.
-------
$ make
/usr/bin/gcc -o obj.i686-linux/skeletons/motor_controller
obj.i686-linux/.build/skeletons/motor_controller/motor_controller.o
obj.i686-linux/emrun/libemrun.a obj.i686-linux/libdev/libdev.a
obj.i686-linux/libmisc/libmisc.a -L/usr/lib -Lglib/obj.i686-linux
-lglib-2.0
obj.i686-linux/libdev/libdev.a(glib_dev.o)(.text+0xb63):
In function `fusd_event_handler':
libdev/libdev/glib_dev.c:545: undefined reference to `fusd_dispatch_aux'
-------
In the example above, we included the `libdev` library, which in turn
referenced a function called `fusd_dispatch_aux`. Using LXR
(http://cvs.cens.ucla.edu/lxr/ident?i=fusd_dispatch_aux) we can
determine that this symbol is a part of `libfusd`. Thus, we must add
`fusd/fusd` to the `local_libs` directive.
I'm including the library and it still doesn't work! What now?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In this case, the problem is usually that the libraries are listed in
the wrong order. The C linker does symbol resolution in a single
pass, and any unresolved symbols encountered while processing one
library must be resolved in libraries that are *yet* to be processed.
Thus, in the example below, `fusd/fusd` must *follow* `libdev/dev`
because `libdev` references `fusd`. Because the order is wrong, some
of the `fusd` symbols are still unreferenced.
--------
$ make
/usr/bin/gcc -o obj.i686-linux/skeletons/motor_controller
obj.i686-linux/.build/skeletons/motor_controller/motor_controller.o
obj.i686-linux/emrun/libemrun.a obj.i686-linux/fusd/libfusd.a
obj.i686-linux/libdev/libdev.a obj.i686-linux/libmisc/libmisc.a
-L/usr/lib -Lglib/obj.i686-linux -lglib-2.0
obj.i686-linux/libdev/libdev.a(glib_dev.o)(.text+0xb63):
In function `fusd_event_handler':
libdev/libdev/glib_dev.c:545: undefined reference to `fusd_dispatch_aux'
--------
In some rare cases libraries can have circular dependencies, and in
those cases you must list one library twice. This does not incur any
additional cost. However, this is very unusual and this case does not
exist in the current EmStar tree.
The easiest way to get the order right is to copy existing `BUILD`
files, that already have `local_libs` populated appropriately.
Advanced EasyBuild
------------------
This section has additional information about EasyBuild for advanced
use and maintenance.
Adding New Platforms
~~~~~~~~~~~~~~~~~~~~
New platforms can be added by modifying the `make/make.platform` file.
This file defines all of the platform-specific values required to
build on the target platform. For the complete, most up-to-date
details, please refer to the `make.platform` file itself. However,
the following description discusses some of the key items:
.`LINUX_ROOT`:
The pathname of the root of the target linux compiler bundle.
.`KERNEL_VERSION`:
The version number of the kernel used by this platform.
.`CC`, `CPP`, `AR`, `LD`, `BINSTRIP`:
The pathnames of the target platform's "cross" `gcc`, `g++`, `ar`, `ld`, and
`strip`.
.`LDFLAGS`:
Default linker flags.
.`KCFLAGS`:
Any flags needed to cross-compile a kernel module, including the include
path for the correct kernel headers.
See more files for this project here