▲ 84 points
back
32 comments
This is Brian. I switched over to Nvidia hardware after building this because of Moore’s law and access to libraries / tools. http://www.parallac.org has some of my machines.
> because of Moore’s law
can you care to elaborate? how is switching to nvidia justified by "becasue of Moore's law"? Genuinely curious.
By the time the parallella Kickstarter shipped, Nvidia shipped the TK1, which was roughly 10x faster FLOPS-wise than a single parallella board. Cost was similar.
kind of tired of calling every small cluster a supercomputer. as someone who has been building actual supercomputers (think top 5-10) for over 25 years, it is very frustrating to hear such misinformation.
encourage parallel computing and scale out - dont call every small project a supercomputer.
Generally agree. The spirit of the term “supercomputer” in my case is tied to a) massive respect for Seymour Cray b) achieving something in the realm of late 90s scale according to top500 and d) parallella’s marketing.
What happened to the 64 core version? And do these chips actually serve a purpose beyond just experimentation and maybe embedded computing?
They failed and the guy behind it joined DARPA. You may have seen their new electronics initiative posted here a few weeks ago - that's run by him.
No, it isn't a supercomputer. These things (whether they be Parallela, Pi, or whatever) are generally at least an order of magnitude too low in every parameter relevant compared with a single node of a real HPC system.
A typical University system from that time might have been have been >~100 nodes with 16 Sandy Bridge cores, ~20 DGEMM GFLOPs and >=2GB ECC memory per core, a QDR Infiniband interconnect (~1μs latency, 40GB bandwidth), and >100TB of Lustre parallel filesystem. Also you could actually manage the hardware.
Don’t be like that. The shorthand here is that this is a novel, massively parallel architecture, much like those in supercomputers. If it had been adopted and scaled up, it could have had interesting applications.
But the parallelism isn't the hard part. The hard part is getting the communication structure right. Scaling up becomes difficult if the complexity of communication is e.g. quadratic in the number of nodes.
if scaling up were so easy, every country in the world would have made 1000s by now. there is a reason scaling up is hard, hard, hard problem. dont make it sound trivial how difficult building actual supercomputer architectures is.
its easy for anyone who doesnt know the challenges to say - yeah, we will just scale it up using duct tape and straws
> The shorthand here is that this is a novel, massively parallel architecture
I'll bite.. I'm actually not at all experienced with supercomputers or HPC clusters, only 1990s proprietary MPP systems, what has been termed "grid computing" [1], and currently-popular systems like Hadoop.
In what way is this novel?
[1] e.g. distributed compilation, semiconductor simulation/test, or similarly embarassingly parallel workloads requiring little formal coordination beyond traditional batch scheduling resource allocation
"Like that" as in trying to educate? Too bad.
This described something isn't "massively parallel" and couldn't scale up. I'm happy with calling the big Sunway- and PEZY-based systems supercomputers and hope that Neo turns out well.
"Like that" trying to tear down a fun little hobby project.
You’re violating the rules of HN.
More precisely:
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
I saw OP threw in technical points where the hobby machine is not like a production HPC box - after issuing a dismissal of the work entirely. HN does best when it gives a hobbyist a good faith appraisal of his/her work. A hobbyist is of course not going to have the resources to do HPC at home - usually.
> I saw OP threw in technical points where the hobby machine is not like a production HPC box - after issuing a dismissal of the work entirely.
Which would make it a non-shallow dismissal, as it provided a number of specific discussion points, which are, in fact, being discussed. The guidelines don't, after all, discourage dismissal (even dismissal in entirety), only shallow dismissal.
> HN does best when it gives a hobbyist a good faith appraisal of his/her work.
This strikes me as a very subject assertion, all around. It's a matter of opinion on how HN does best, and it's a matter of opinion what constitutes a good faith appraisal, especially as it relates to a hobby.
That said, I might agree for a "Show HN" post, but not for a repost of something 4 years old, with the arguably clickbaity (and patently, unarguably controversial) unqualified term "supercomputers" in the title.
> A hobbyist is of course not going to have the resources to do HPC at home - usually.
If not, I'm left to wonder, what's the point? Where is the performance "high" or the computing "super"?
I could somewhat wrap my brain around a hobbyist building a very small (in node count) HPC cluster as a testbed. I could also understand if the goal were something like cost-performance efficiency improvement (for purchase and/or operating cost). Is that what's going on here? I didn't get that from the article, and only one comment claimed the architecture is "novel" without specifying in what way.
I didn't even see anything to suggest scalability/modularity that's easier than "real" HPC, but that could be because I missed it among the emphasis on handmade construction.
> a QDR Infiniband interconnect (~1μs latency, 40GB bandwidth)
This is the key aspect of actual HPC clusters that appears to be missing from the pretenders.
Some of the sibling comments point out that these scaled-down versions can be useful for testing in advance of deploying to (or building in the first place) the real cluster. However, that only makes sense if everything is scaled reasonably proportionally, and that can be tough with I/O, if latency and not just bandwidth, is important.
I can imagine a tiny platform like a raspberry pi being credibly interconnected with gigabit ethernet, but even a vaguely modern low-end x86 cpu can saturate it (and maybe even 10GE), never mind the (100x?) latency compared to IB.
This is the key aspect of actual HPC clusters that appears to be missing from the pretenders.
That completely depends on your workload. For example, we did (natural language) parsing on thousands of CPUs on one of the large European computation grids. This work was largely CPU bound, the input and output data is relatively small, so the interconnect speed is not a big factor. The problem is embarrassingly parallel, so the primary consideration is having enough CPUs with a reasonable amount of memory.
When we say "HPC" generally, we're talking about being able to run computation that isn't embarrassingly parallel. The bulk of work on typical general purpose systems tends to be DFT-type materials science and finite element/CFD engineering. Obviously if you have a special purpose system you can made different trades-off.
To be fair, you original comment did use both "supercomputer" and "HPC". I'd argue that, although in most people's minds, the two are indistinguishable (perhaps like the words "envious" and "jealous"), but carry a unique significance to practitioners.
That said, any lay definition of supercomputer would seem to require actual performance (e.g. high FLOPS rate), which trivally supports the initial assertion.
One can call something like the Folding@Home project a distributed supercomputer, but it's not that interesting, especially to build such a system, for anything embarrassingly parallel.
To many practitioners (people using HPC rather than people designing HPC), HPC means an aggregation of computing resources to deliver orders of magnitude of higher performance than one could get from a single workstation.
Obviously if you have a special purpose system you can made different trades-off.
It should be obvious, but the post I was reacting to was claiming that it cannot be 'real HPC' without high-speed interconnects. It depends on what you are a cluster/HPC facilities for. Typically institutions (such as universities) have different facilities with different trade-offs.
> HPC means an aggregation of computing resources to deliver orders of magnitude of higher performance than one could get from a single workstation.
> the post I was reacting to was claiming that it cannot be 'real HPC' without high-speed interconnects
That was me, and that's still my point. The perception of users that have the embarassingly parallel workload isn't particularly meaningful. We already had the pretty generic terms "grid computing", "distributed computing" and "cluster".
What's the point of nomenclature like "HPC" if it describes something that's indistinguishable from that single workstation merely multiplied by some large number, somehow attached over the Internet?
If it's not "real HPC", then what is a better term of art to describe the sort of computing that can deliver high performance to workloads that include ones that are not embarrassingly parallel?
Exactly, on the scaling, but even then you probably won't see potential problems with, say, deadlock, that may hit you on a real system. You can't stress latency enough, as you'll now if you've ever seen something like cp2k limping over TCP/Ethernet.
By the way, if you want to simulate performance of real coes on real systems (as you should), you use programs like dimemas, simgrid, and others with real or projected parameters.
Ok, a poor man's supercomputer then ;). I think such projects are great to experiment with parallel computing, get experience with ARM, and just plain fun.
IIRC, you can test your OpenMPI workloads on a desktop cluster before you push them to your really big machine.
Various super computer labs are using single board computer clusters of similar node size (but obvious less compute) in order to simulate the parallel compute issues without consuming the main cluster. It is just another step in the testing process for complex software that might very well waste cluster resources.
It's not a number cruncher, but it emulates the same issues you see on an HPC cluster. To the point it's actually used for that.
https://www.extremetech.com/extreme/259553-750-raspberry-pi-...
Having said that, a cluster of generic boxes doesn't really deserve to be placed in the same category as machines engineered from the ground up to be a supercomputer.
Is the infiniband 40Gbytes/s or 40Gbits/s? I seem to recall that it is 40 gbits/s?
It would be 40 gigabits per second, and I hadn't even noticed the capital B (which I assume was a typo) until you pointed it out.
I was about to say that this is a consistent convention for networking to use bits instead of bytes, but I realized this breaks down with something like IB that has been used as a bus-like interconnect and a storage network/fabric (both areas where bytes are the more common convention in rates).
Even today, 40 gigabytes per second would be pushing the I/O capacity of a typical Xeon, though not a 128-lane Epyc, and I'm not sure about POWER9.
With 200Gb/s IB with both bandwidth and latency on the same OOM as other-socket RAM, it's also pretty close to pushing that capacity, which is, of course, the whole point, I suppose.
I only see a blank page for this blog, due to NoScript, but what's weird, is I can't disabled NoScript for this domain at all, and it says "This is a privileged page, whose permissions cannot be configured."
I've never been to this site before, and never seen any other domain do this to NoScript.