Has anyone tried RAM based filesystems to speed up c++ compilation?

I need faster incremental builds, meaning: under 2 seconds. Is this possible? I already tried precompiled headers which did nothing. I have a fast desktop machine with 16gb ram and another quadcore machine with 4gb ram that I can use for this.
Wouldn’t I need to put the entire OS on ram to make this work? All the gcc headers and libraries live in weird os locations right?

On Mon, 18 Mar 2013 20:46:02 +0000, Maxxi12 wrote:

> I need faster incremental builds, meaning: under 2 seconds. Is this
> possible?

Depends on what it is that you’re building. If you’re talking about
building gcc or the Linux kernel, no, you won’t get a sub-2 second
compile no matter what you do.

If you’re talking about compiling “hello world”, then you should get that
from a regular hard drive without any problem.

It would help if you specified what it is that you’re trying to build.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

True, I get 2 second builds from small examples, but 2 seconds is already the upper limit of what I find acceptable iteration speed. As soon as I hit 3 classes which have to interact with each other and include a database library with its headers (MongoDB in my case), suddenly the build times are at 4-5 seconds. It’s really not a huge project, but at a few files that include each other more or less, it starts getting annoying to wait 5 seconds. The compile times on c++ is the only reason I’m not using it for everything.

A couple of thoughts:

  1. Linux caches the filesystem for you. I would bet a brass nickel that
    the majority of things involved in slowness (all libraries, headers from
    other products, and everything about gcc) are already cached in RAM, so
    having your entire system loaded into a “RAM drive” is probably already
    happening as much as necessary. This is part of the reason why
    applications load faster the second time than the first. Anyway, all of
    these are likely to be bigger than the code being compiled for most
    smaller projects.

The actual files being compiled are also probably cached since, right
after you edited them, they were saved to disk and probably cached in the
process. I do not know how to prove this, but unless you are dealing with
many MB of source files changing every time I am going to assume it.
You’re welcome to try to copy (not move) these somewhere into a “RAM
drive” (like /dev/shm for example) to show otherwise. I doubt you will
see much benefit, and that’s probably because you’re already on the low
end of possible compile times for your processor.

Other options may help; you could decrease GCC’s optimizations (I wouldn’t
do this, personally) and maybe that will help. You already mentioned
pre-compiled headers which should help, assuming header compilation takes
a significant amount of time (you’re talking about two seconds; don’t
count on it).

If I were you I’d setup a silly little script that loops indefinitely
while you are developing and basically does the following:

Code:

while 1 ] ; do
INTERVAL=5
#find source file modified in the last ${INTERVAL} seconds.
if filesModifiedWithinINTERVAL ]; then
recompile
fi
sleep ${INTERVAL}
done

You’re left to implement the code for ‘recompile’ as well as setting the
‘filesModifiedWithinINTERVAL’ boolean. At the end of the day you run this
and every time you save, within a few seconds (defined by you, and about
the time it’ll take you to run your program again for testing) things are
rebuilt.

Good luck.

On 2013-03-19 07:36, Maxxi12 wrote:
> It’s really not a huge
> project, but at a few files that include each other more or less, it
> starts getting annoying to wait 5 seconds. The compile times on c++ is
> the only reason I’m not using it for everything.

I once tested compiling speeds, and it turned out that if you put the
sources and the destination into a reiserfs partition the make process
is much faster (compared with ext3 or xfs). I guess that a tmpfs would
be even faster, but I did not try that: the kernel is too large.

(and the kernel filesystem cache doesn’t make much of a difference when
building, say, a kernel, I’m afraid. Much too big for the ram available
on normal systems)


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On Tue, 19 Mar 2013 06:36:01 +0000, Maxxi12 wrote:

> True, I get 2 second builds from small examples, but 2 seconds is
> already the upper limit of what I find acceptable iteration speed. As
> soon as I hit 3 classes which have to interact with each other and
> include a database library with its headers (MongoDB in my case),
> suddenly the build times are at 4-5 seconds. It’s really not a huge
> project, but at a few files that include each other more or less, it
> starts getting annoying to wait 5 seconds. The compile times on c++ is
> the only reason I’m not using it for everything.

It might be necessary to just learn some patience. Not everything is
“instant”, and compiling is one of those things that isn’t.

Heck, I can remember building GCC on a Sun 2 workstation - it took an
evening to run.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On Tue, 19 Mar 2013 15:26:25 +0000, Jim Henderson wrote:

> It might be necessary to just learn some patience. Not everything is
> “instant”, and compiling is one of those things that isn’t.

To add to that, I/O may not be your bottleneck. Compiling tends to be
very CPU-intensive as well.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

Unfortunately this wouldn’t help, first of all I have to save files before I can look for a filemodified event, and it still wouldn’t help me with quick changes + f5 run because there is almost no wait at all between the change and the run. In addition to that, I’d have to catch compilations cycles that are already running in Qtcreator and wait for them to complete, then issue another f5 run call. It would essentially require me to implement a qtcreator plugin.
This is all way too much work, for very little actual benefit.
When you look at distcc, the scores seem pretty **** bad: only 300% improvement with 38 machines, and that with large projects.
benchmark results for distcc

I’m sure that if I combine it with fully ramloaded os and distcc 3 machines, I could get a nice speedup in compilation. It’s a lot of work to setup, and a “nice” speedup isn’t enough, I need consistent sub 2 seconds…

On 2013-03-19 16:26, Jim Henderson wrote:
> It might be necessary to just learn some patience. Not everything is
> “instant”, and compiling is one of those things that isn’t.

When I started using Linux, more than a decade ago, I was surprised at
the speed of compilation in Linux: much slower than in MsDos, compared
with Borland C, which is what I used at the time professionally.

And in MsDos Borland C was slower than Borland Pascal.

I don’t say this to start a flame war: it is simply a fact that
surprised me a lot.

(that Borland Pascal was faster than Borland C was no surprise, the
reason is known: what they called “smart linking”).


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On Wed, 20 Mar 2013 00:28:06 +0000, Carlos E. R. wrote:

> On 2013-03-19 16:26, Jim Henderson wrote:
>> It might be necessary to just learn some patience. Not everything is
>> “instant”, and compiling is one of those things that isn’t.
>
> When I started using Linux, more than a decade ago, I was surprised at
> the speed of compilation in Linux: much slower than in MsDos, compared
> with Borland C, which is what I used at the time professionally.
>
> And in MsDos Borland C was slower than Borland Pascal.
>
>
> I don’t say this to start a flame war: it is simply a fact that
> surprised me a lot.
>
> (that Borland Pascal was faster than Borland C was no surprise, the
> reason is known: what they called “smart linking”).

Oh, sure, it’s gotten much faster, for a number of different reasons.

I build kernels for my router (again running openWRT on new hardware
now), Rockbox firmware for my iPod (which doesn’t actually get a lot of
use any more), and even firmware for my Android phone.

The time is measured in tens of minutes at least for each of those.

If it compiled in 2 seconds (heck, even 5 seconds!), I’d be amazed.

But of course the time it takes to compile depends on how much code
you’re compiling.

The gcc compile I mentioned before ran overnight - but gcc is something
of a special case (or was, I haven’t tried building it recently) because
it used the Sun compiler to do its first build, then it would build
itself using its first build; then - as I recall - it would do an
optimization build using its self-build. The end result was gcc built by
gcc built by cc.

I find it honestly surprising that a 5 second build is “too slow” for the
OP, regardless of the code being built. Spin your chair around once or
twice while the build is running. Get up and walk around for a few
seconds while it runs.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

Carlos E. R. wrote:
> (that Borland Pascal was faster than Borland C was no surprise, the
> reason is known: what they called “smart linking”).

I don’t know anything about that. I do know that Pascal syntax was
specifically designed for ease of parsing - LL(1) - whereas C was not. C
requires a more complex compiler.

On 2013-03-20 11:54, Dave Howorth wrote:
> Carlos E. R. wrote:
>> (that Borland Pascal was faster than Borland C was no surprise, the
>> reason is known: what they called “smart linking”).
>
> I don’t know anything about that. I do know that Pascal syntax was
> specifically designed for ease of parsing - LL(1) - whereas C was not. C
> requires a more complex compiler.

The linker was also very special. Borland Pascal did not use standard
object files, but a format of their own, not standard, with several
advantages. I do not remember all of it, but besides strict syntax
checking, it allowed for linking only of the functions and variables
that were actually used, the rest was ignored. This made the resulting
binaries much smaller, but also faster to link. They called this “smart
linking” with a lot of big lettering on their publicity.

Another trick was the integrated “make” vs “build” procedures. On
complex projects, it was clever enough to consider differently the files
(units) where the external API changed or only the part beyond the
“implementation” section of the unit. If you only changed the
“implementation” section, units depending on it were not recompiled.

They cleverly used (and redefined) the language to their advantage, the
compiler was very fast even on complex projects.

And the Borland C compiler was also fast, orders of magnitude faster
than the “standard” C compiler that Microsoft provided (not a surprise).
But it was also faster than GCC in Linux building similar projects, and
this is something that disgusted me when I switched to Linux.

A trick we used at the time (1995…98) to minimize the size of binaries
(which was very important, memory was limited), was to put only one
function per file, instead of dozens of functions per file. The
rationale is that if you only need one function, the entire file with
all the functions is linked, used or not (I don’t know if this happens
also in Linux, but I think it is).

The downside was that projects were spread over dozens or hundreds of
small files, which had to be found and compiled, and later linked as
necessary. This is much slower than a few large files.

And this high number of small files, such as you find in the kernel
sources, is the reason that making the kernel is much faster if the
filesystem is reiserfs instead of ext3.

Do the measurements if you don’t believe me! I did, years ago. :slight_smile:

I would be curious to know how differently ext4 behaves now.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On 2013-03-20 04:50, Jim Henderson wrote:
> I find it honestly surprising that a 5 second build is “too slow” for the
> OP, regardless of the code being built. Spin your chair around once or
> twice while the build is running. Get up and walk around for a few
> seconds while it runs.

The time you wait for make to finish is wasted time, if you are paid for
it :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On Wed, 20 Mar 2013 17:53:05 +0000, Carlos E. R. wrote:

> On 2013-03-20 04:50, Jim Henderson wrote:
>> I find it honestly surprising that a 5 second build is “too slow” for
>> the OP, regardless of the code being built. Spin your chair around
>> once or twice while the build is running. Get up and walk around for a
>> few seconds while it runs.
>
> The time you wait for make to finish is wasted time, if you are paid for
> it :slight_smile:

If it’s measured in seconds, then it’s not that much “wasted time”; but I
also don’t know many professional developers who don’t have multiple
systems.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2013-03-20 23:46, Jim Henderson wrote:
> If it’s measured in seconds, then it’s not that much “wasted time”; but I
> also don’t know many professional developers who don’t have multiple
> systems.

When I programmed for money, I had one or two computers. Having a
hundred would not have made the slightless difference: I still had to
wait for that single compile to finish and test the result.

And I had to wait minutes, not seconds. Compile, test, change,
recompile. With thinking in the middle :slight_smile:

Lab equipment, process automation and measurement… the test machine
was single. Not every professional programmer works in multiteam
projects with thousands of dollars :slight_smile:

(Hey, I had to bring my food from home in a tapper: not enough money to
go to a cafe or restaurant. Not every developer is lucky to be well
paid. I have the feeling this has not changed much)


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)