▲ 70 points
back
14 comments
COMPUTE!'s Gazette published a three part series on an assembly language neural network in 1990 for the C64, which is infinitely faster than this, but much more opaque. The article was pretty thorough as I recall, though.
John Walker said (in 1987):
> If you walk up to a computer and ask it “what is the name of the famous bridge in the same state as Disneyland” you won't have much success, but most people will immediately answer, “The Golden Gate Bridge”.
I prompted GPT-2 on talktotransformer.com with that question. The first two times it gave me nothing coherent (a made-up URL about Minnesota, and something about Georgetown), but the third time, it said:
What is the name of the famous bridge in the same state as Disneyland?
This is a double fact: it is a California Heritage Bridge, a nickname invented by the California Turnpike Authority and Santa Clara County Transportation Authority (whose name is, coincidentally, the original name of the Carson City Parking Authority).
How did the first hippie, Kesey, travel across the Golden Gate Bridge?
In Kesey's famous classic "One Flew Over the Cuckoo's Nest" he ruminates the great flight across a river, and then some.
This is almost entirely false — there is, for example, no such thing as a "California Heritage Bridge", and I can almost guarantee you that the Carson City Parking Authority, if it exists, was never called the "Santa Clara County Transportation Authority". However, GPT-2 clearly did manage to make the association with the Golden Gate Bridge, as well as several other things about California.
My attempts to rephrase the question in a more GPT-2-friendly way, such as "There is a famous bridge in the same state as Disneyland, named", were not successful, although they did confabulate an entertainingly large variety of bridge names.
I tried a similar experiment on GPT-2 a few days ago, giving prompts like "if I drop a stone, it will" or "wood floats in water while a stone". Because that thing definitely has some knowledge of the real world, otherwise it wouldn't be able to produce chunks of semantically plausible text.
The results were disappointing though, it mostly rambles and only occasionally completes the sentence correctly. I wonder if the engine could be tweaked to produce more semantically consistent statements- maybe as a specific "style" (as it's very well versed in reproducing writing styles and formats).
Yeah, it seems like the question format led it to imitate the "style" Q&A web pages which are mostly infotainment, but mostly don't bother to mention facts that are so well known as the existence of the Golden Gate Bridge. Imitating IRC "style" produces people ignoring the question:
> <bob> wut is the name of the famous bridge n the same state as disneyland
> <DocDrew> small
> <DeathTrip> lmao, you "drew" this up
> <DeathTrip> 'sup DocDrew
> <drewk> Wasn't James Cameron the guy to make the first ever 3D movie?
> <drewk> :p
> <drewk> Also the guy who claimed to invent digital video in the first place.
While adding timestamps and a space for @ leads it to imitate specifically #bitcoin:
> 19:33 < bob> wut is the name of the famous bridge n the same state as disneyland if you wanna know 17:19:37 <nalixor> this is actually actually helpful. 17:19:47 <nalixor> what?! 17:20:12 <Pnutbot> We should note that I'm not an expert on the US government nor the wallet industry, but as an alts-trader I'd like to see this explored thoroughly. 17:20:21 <Pnutbot> there might be people with other knowledge like
Alternatively:
> 19:33 < bob> wut is the name of the famous bridge n the same state as disneyland?
> 19:34 < warren> Old Alaska?
> 19:34 < warren> or Nome?
> 19:35 < warren> I'm not familiar with that town
> 19:36 < warren> http://en.wikipedia.org/wiki/Alaska_state_homepage#Social_ex...
> 19:36 < warren> sometimes is chosen by the change the spelling of homeport
Imitating the "style" of a play results in a back-and-forth dialogue, but an unhelpful, dramatic one:
> John: What is the name of the famous bridge in the same state as Disneyland?
> Johnathan: Uh, no. No, that's not what this is.
> John: Okay.
> Jonathan: That is not a bridge. What is this? A super bridge?
> John: Uh, yeah, it's a bridge.
> Jonathan: And now we got them in our fire truck… the bridge.
> John: With their fire trucks.
> Jonathan: And we got them in our fire truck, and we got the roof lifted up
Imitating the style of a screenplay results in another dramatic but unhelpful conversation about bridges and murder:
> JIM looks thoughtful.
> JIM
> (finally)
> What is the name of the famous bridge in the same state as Disneyland?
> TRACY
> No biggie. Like the bridge for Opus--
> INT. ANDY'S CAR - NIGHT
> JIM and TRACY arrive at a quiet house.
> TRACY (cont'd)
> ... it's this house in a country where you won't get pulled over.
> JIM
> Or shot.
> TRACY
> And where you are a reasonably happy
Formatting it explicitly as a Q&A leads to especially unhelpful results:
> Q: What is the name of the famous bridge in the same state as Disneyland?
> A: ???????
> Q: If I had an extra $1 billion in the bank, would I donate it to my state to fund a road?
> A: ???????
> Q: If I knew of a company that I was not allowed to talk about because of internal corporate policies, what company would I go to?
> A: ???????
> Q: What do most big brands have in common?
> A: ???????
Imitating a numbered Q&A format is only slightly more effective:
> 5. What is the name of the famous bridge in the same state as Disneyland?
> That bridge is called Yosemite Falls and if you want to know its name then click here to view Google Map for this spot.
> 6. I have found a claim on Google that claims that this park is a National Park. Is this true?
> No, it is not true as this park is listed as a County Park. Although this park is considered by many to be a National Park it is completely within the boundaries of California. The park has plenty of hiking trails that could actually qualify it
Using the beginning of the Wikipedia article about Disneyland, plus a slight prompt, was effective:
> Disneyland Park, originally Disneyland, is the first of two theme parks built at the Disneyland Resort in the same state as various famous bridges, such as
After a few tries, GPT-2 completed this with:
> the Golden Gate Bridge, and the San Francisco-Oakland Bay Bridge. The park was first opened to the public on July 15, 1955, and was designed by Robert E. Iger. In 2005, an expansion expansion was completed, adding thirteen new roller coasters and 13 new
Even still, it often generates human-sounding nonsense:
> the Hollywood Bowl and Golden Ledge. The Hollywood Walk of Fame is an atlas-style listing of the names of well-known celebrities in the Walt Disney Company who have accepted the honor of being among the park's heroes. The current structure was built in 1939-1940 and
And of course even when it gets it right it gets it wrong:
> the Golden Gate Bridge and the San Francisco, USA bridges.
> Worlds Fair of America and World's Fair of Texas [ edit ]
> In 1969, Disney established the World's Fair of America, a giant mechanical thing and show meant to feature large scale attractions like a Big
A thread from 2017: https://news.ycombinator.com/item?id=15882683
Wow, a blast from the past. I ported this program to 6502 assembly and added more features, and used it for a school project.
Then I read all of the articles on neural networks claiming they were a dead end and went off to other things :)
I love historic pieces like this and it's wonderful to see these early experiments.
It's really easy to get this to run! Download VICE Disc Image of this Program, unzip it (to get the .d64 file), then go here: https://virtualconsoles.com/online-emulators/c64/ and drag in the .d64 file to see it run.
Another note, for those not familiar with the brilliant author: "This site is developed and maintained by John Walker, founder of Autodesk, Inc. and co-author of AutoCAD. "
Has anyone rewritten this in something modern? Probably less than 100 lines of Python, for example. Can skip load/save, .... I think he uses the C64 ROMs to get the bitmaps. All those pokes around 1024 are writes to the screen RAM.
# READ Bitmap for selected character from C64 ROM
310 k=asc(a$)
320 if a$>="0" and a$<="9" then k=k+64:goto340 # process numbers 0..9
330 if a$ < "a" or a$ > "z" then 500 # Maybe a function key
340 gosub 970:print "fetch ";a$
350 l%=0
360 k=(k-64)*8+53248
370 poke56333,127:poke 1,peek(1)and251
380 fori=0to6:poke49408+i,peek(k+i):next # Copy character's image from character ROM (53248) to RAM
390 poke 1,peek(1) or 4:poke 56333,129
#
# Look up bits that make character 6x7 = 42 values
# Lighted dots are stored as 1 and background dots as −1
400 for i = 0 to 6
410 j% = peek(49408+i)/2 # byte value - divide by 2 is shift right. Removes first bit
420 for k=1 to 6 # 6 wide
430 l%=l%+1 # counter 1..42
440 f1%(l%) = -1 + (2 * (j% and 1)) # 2 * (1 or 0) => field[counter] = 1 or -1
450 j%=j%/2 # Shifting right again to get the next bit in first position for the AND operator. xxxx xxx1
460 next k
470 next i
# Function keys
510 if j%=1 then gosub 1000:goto 280 # train()
520 if j%=5 then gosub 1080:goto 90 # print_matrix()
530 if j%=2 then gosub 1210:goto 280 # randomize_ten_percent()
540 if j%=6 then gosub 1680:goto 280 # clear_memory()
550 if j%=3 then gosub 1290:goto 280 # recall_pattern()
560 if j%=7 then print chr$(147);:close15:end
570 if j%=4 then gosub 1800:goto 90 # save_to_disk()
580 if j%=8 then gosub 1990:goto 90 # restore_from_disk()
1000 rem train on pattern in f1% def train()
1010 gosub 970:print "training"
1020 for i = 1 to 42
1030 for j = 1 to 42
1040 m%(i,j)=m%(i,j)+f1%(i)*f1%(j) # matrix[i][j] = matrix[i][j] + field1[i] * field1[j]
1050 next j
1060 next i
1070 return
1210 rem randomise 10 percent of f1% def randomize_ten_percent()
1220 gosub 970:print "random"
1230 for i=1 to 42
1240 if rnd(0) > 0.1 then 1260 # Skip > 10%
1250 f1%(i)=-f1%(i) # negate
1260 next i
1270 gosub 750 # update field f2 on screen
1280 returnThe "poke56333,127" is disabling the background keyboard reading IRQ by disabling the "IRQ on timer out" function of the CIA 1 used to trigger it.
This is needed because the "poke 1" that follows maps in the 4K character ROM over the I/O devices at D000-DFFF; if you don't do this the system would freeze.
I got my Master's Degree in CS in 1988. For one project, I did a Neural Network on an IBM-PC, in Turbo Pascal, which is barely a 16-bit computer, back in this year. We were training it for various logic functions, and very simple "simulated retina" problems with patterns on 16x16 grids.
Barely a 16-bit computer? The IBM PC's CPU could address a mebibyte of RAM — that's 20 bits! In a single process! You could actually put 640 KiB of RAM in it, I'm pretty sure. A computer that was barely 16 bits would at most have a 16-bit address space, maybe less, though I don't know of any that took that option.
The 8088 had an 8-bit data bus, 16-bit registers, with a segmented scheme that would allow 20-bit memory addressing.
The 8086 had a 16-bit data bus. See https://en.wikipedia.org/wiki/Intel_8088
Usually the "bitness" of the computer is the width of the general registers. Most 8-bit computers had a 16-bit address counter (for 64K of memory), with 8-bit general regisgters.
Yes, all of that is correct, although I think you have conflated "address bus" with "program counter".