Asmttpd – Web server for Linux written in amd64 assembly

AFAIK Linux doesn't use zero-terminated strings anywhere in its syscalls, or at least not in those like write, where you pass a size alongside the buffer.

I'm surprised it doesn't do length-prefixed strings with a null terminator anyways. Makes a whole lot of things easier.

Well, for a HTTP server, I don't have a specific idea... But in general, the fun part would be trying to come up with string representations that are optimized for the particular application.

The original 1984 Elite computer game is famous for its huge galaxy full of planets. Each of them had individual names and descriptions such as "Lave is most famous for its vast rain forests and the Laveian tree grub."

Yet those strings were never stored as plain strings. The game had to run in 32kB of memory, so almost all strings were stored in a tokenized form and expanded using a pseudo-random number generator:

http://wiki.alioth.net/index.php/Random_number_generator

That article shows how the planet description strings were stored and reconstructed on the fly. The base representation for the aforementioned description of planet Lave was only a handful of bytes: "\x8F is \x97"

So I think Elite is a pretty good example of an application written in assembly that didn't have anything like a generic string type.

As others have said, have a length field in with the string. This also has advantages other than making buffer overflows a lot harder, such as making string copying faster and easier. For example, let's have a skeletonized view of a normal string copy routine in assembly (disclaimer: my assembly is rusty, so this may not be completely right. void where prohibited):

        push rax ; save our registers
        push rdi
        push rsi
        mov rsi, location ; get the pointer to the right place
	mov rdi, destination
		
    beginning:
        mov rax, [rsi] ; copy contents to the register so we can compare
       test rax, rax ; compare our source to itself, if it's zero, it'll set a flag
       jz done ; we're done
       movsb ; copy the byte, increment the rsi and rdi registers
      jmp beginning
	
    done:
        pop rsi ; restore our registers
        pop rdi
        pop rax
		

In contrast, with a length parameter, we can do

        push rcx ; using a different register here
        push rdi
        push rsi
        mov rsi, location ; same as before
        mov rcx, length ; moving our length into the counter register
        mov rdi, destination

        cld ; okay our first change. Clearing the direction flag so the copy
            ; goes from the first byte to the end
        rep movsb ; it'll repeat cx times the movsb command, and then carry on
	    
        pop rsi ; restoring our registers
        pop rdi
        pop rcx

Having the length of strings means you can have much more concise code. It makes loops easier, makes your code cleaner, and in some environments, gives a speed boost.

Turbo/Borland Pascal used a NUL-optional string format of length (byte IIRC) followed by data. [0]

Java IIRC also uses a length-oriented format for string constants in .class files. [1] It's been a while since I wrote a .java to MIPS asm compiler in C++ from scratch (don't ask).

This is because real-world strings may contain 0 to N NULs and escaping them is too much of a PITA for serialized formats, so it's easier and common to do things like TYPE LENGTH DATA de/serialization. For modern, efficient binary de/ser, check out binc and msgpack [2,3].

0: http://math.uww.edu/~harrisb/courses/cs171/strings.html

1: https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.ht...

2: http://msgpack.org/

3: https://github.com/ugorji/binc/blob/master/SPEC.md

Could have a pointer, length pair. That's how it's done in some non-C languages.

I remember one program that had a string print subroutine that took zero args. It just used the return address from the stack to grab the nul-terminated string immediately following the JSR/CALL instruction. It then patched the return address on stack to return just after the nul.

Bad for storing data in .text, but still a neat hack that shaved whole tens of bytes off the program size.

> Oh, I bet someone writes httpd in vhdl or verilog

I would love to see that.

That already exists, Node.js.

To avoid overhead people implement specialized compilers suited for the task at hand. If anything, going down to C, and especially assembly will hurt performance as low-level code is much harder to optimize for obvious reasons. Above all of that real-world performance comes from proper system-level design, not micro-optimizations, and using a low-level language (be it C, C++ or assembly) will prevent one from quickly iterating over different ideas.

Only if you use a completley naive compiler, any level of optimisation moves from being a literal translation

> call it "C" for convenience

As distinct from C, I take it.

Just because it's asm doesn't mean it's fast, sure. Just because it's C doesn't mean it's faster than an implementation in your favourite scripting language.

Having said that, compilers do pretty badly on C-to-simd optimisation. The best you get is loop vectorization if it's really simple logic. You can usually get some pretty good wins there. The fact you lay out your memory for simd usually is a win all of its own due to cache prefetching even if you don't actually use any simd instructions. Compilers need heuristics to manage cache when you know what you're trying to do, (eg when should it use non-temporal writes, for example?) Fast C code is written while having a really clear mental model of the underlying architecture and the assembly that the C will produce with -O3 (or whatever flag is relevant to your compiler) and then checked with -S or objdump -D, profiled with callgrind/cachegrind, perf, rdtsc etc...

The compiler really can't "Do it for you" You /can/ use a compiler as one of your tools when /you/ do it. As Randy Hyde points out you can always beat the compiler because you can use its generated assembly language in every case you can't beat, so the absolute worst you get is a tie.

So yeah, you can totally smoke clang, Intel, microsoft and gnu C compiler and get paid something for doing it in certain industries too. :-)

Mike Acton being aggressively opinionated on the subject, but the lecture is really good (despite/because of) the bits you'll disagree with and the manner he'll rub you the wrong way. https://www.youtube.com/watch?v=rX0ItVEVjHc

That's often true. Really often true. But sometimes, a GOOD assembly programmer can beat a C compiler by a LOT. https://news.ycombinator.com/item?id=8508923 "Hand Coded Assembly Beats Intrinsics in Speed and Simplicity"

Doesn't look like it supports HTTP pipelining:

    [trent@ubuntu/ttypts/4(~s/wrk)%] ./wrk -c 1 -t 1 --latency -d 5 http://localhost:8080/Makefile 
    Running 5s test @ http://localhost:8080/Makefile
      1 threads and 1 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency    35.00us    0.00us  35.00us  100.00%
        Req/Sec    10.00      0.00    10.00    100.00%
      Latency Distribution
         50%   35.00us
         75%   35.00us
         90%   35.00us
         99%   35.00us
      1 requests in 5.10s, 1.57KB read
    Requests/sec:      0.20
    Transfer/sec:     314.55B

Note the 1 request. For 1000 clients, it's only doing 1000 requests:

    [trent@ubuntu/ttypts/4(~s/wrk)%] ./wrk -c 1000 -t 1 --latency -d 5 http://localhost:8080/Makefile
    Running 5s test @ http://localhost:8080/Makefile
      1 threads and 1000 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency   307.93ms  552.88ms   1.63s    87.10%
        Req/Sec   414.29    439.07     1.34k    85.71%
      Latency Distribution
         50%    4.11ms
         75%  407.63ms
         90%    1.63s
             99%    1.63s
      1000 requests in 5.01s, 1.53MB read
      Socket errors: connect 0, read 41, write 0, timeout 0
    Requests/sec:    199.79
    Transfer/sec:    312.96KB

this is really really fast.