back
12 comments
multi-threaded programming is a lot harder than it looks. that's true of a number of things, like opengl, but most programming subjects can be mastered by the usual bumble-along-and-break-things method that i suspect many of us use.

that's not so true for multi-threaded problems. newbies are likely to create bugs that only show up under heavy load and are nearly impossible to recreate and debug. it was enough to scare me away from multi-threaded programming for many years.

now i know that the solution is extreme discipline. don't share objects between threads, create locks sparingly, have threads communicate with each other only through queues or other well-defined interfaces, etc.

but my knowledge of the subject is pretty old-skule. every now and then i read something about memory barriers, lock-free programming methods, and so on, that make me think other people might have a better way.

i think dataflow programming is a pretty good angle of attack
I'm currently taking a course with Professor Uzi Vishkin a parallel computing visionary. His approach to programming mutli-cores is quite impressive. The system he has devised, while complex under the hood - is relatively simple to the programmer.

Essentially, in Professor Vishkin's model the program alternates between parallel and serial mode as needed. Under the hood, it is determined what processors handle what threads etc.

Have a look at: http://www.umiacs.umd.edu/~vishkin/XMT/index.shtml.

Parallel will be to serial, what the car was to the horse.

...which reminds me -- one of these days I'll have to learn Erlang.

"Given that Erlang programs are naturally written in a concurrent manner, most programs require no modification at all to take advantage of SMP."

http://www.erlang.org/faq/faq.html#AEN625

You get far more than multicore scalability for free. You get distributed fault-tolerance for free. It's kind of awesome.

With some work on its string handling and "talking to the rest of the world" stuff, Erlang could be really awesome.

Threads may be the most confusing way in existence to take advantage of multiple cores. Berkeley's Edward A. Lee has a lot to say about it in his paper "The Problem with Threads" ( http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.... ).
Pardon my ignorance, but if you program in multi-threads using a high level language what assurance do you have that the code will run in parallel on the chip level/lower level. Assuming that you're running a multi-core processor, that is.
Basically, it depends on your language implementation, and whether its implementation of threading is based on OS/hardware threads (and therefore multiprocessing) or on software ("green") threads. Often if OS threads are supported, it's actually a hybrid solution.

There are a number of Common Lisp implementations that support MP, often on only some target OSes, and there are a number that don't. To my knowledge, (I'm not a Python hacker!) vanilla Python doesn't support MP, whereas I suspect that JPython and IronPython support the MP infrastructure that their underlying VMs provide.

Software written for green threads may often fail mysteriously on OS thread implementations. Likewise, green-threaded software may need explicit scheduling, which code written for OS threads will generally not be doing.

In user space? None, but if the processors aren't doing any other work and your threads are the only available runnable threads then they're likely to get run. As cores become more numerous, the problem will be keeping them occupied. That's why Intel and AMD want you to start writing multi-threaded code. NOW! You're making them look bad with all those cores doing nothing. :)
They (Intel and AMD) keep saying that. If there's an elite few, it's because there are damn few jobs that require that as a primary skill. Other skills are more important as primary skills and if you have those you don't need threading skills. You'd be better off investing in those other skills.
This is probably true,but if you are a server side programmer, then its useful to keep in mind that programming for multiple processors will be a necessary skill to know in the future and learn whatever little's possible.
That future remains elusively out of reach. I've been doing multi-threaded programming for over two decades and lock-free algorithms nearly as long and it's just as marginal now as it was then.