▲ 85 points
back
57 comments
The go-to source for randomness in Linux is /dev/urandom, is it not?
I’m not sure the best way to make use of it as a source of random decimals — there are surely a dozen ways to do so — but this works:
printf %d 0x`
head -c 1000 /dev/urandom
| shasum
| head -c 8
`
> The go-to source for randomness in Linux is /dev/urandom, is it not?
AFAIK it's the go-to source of randomness in any modern-ish unix.
> I’m not sure the best way to make use of it as a source of random decimals — there are surely a dozen ways to do so — but this works:
`od(1)` will do binary data to decimal:
> head -c4 /dev/urandom | od -An -vtu4
4119929390
> head -c4 /dev/urandom | od -An -vtu4
3540752584
> head -c4 /dev/urandom | od -An -vtu4
998126897There are several scenarios where a cryptographic-quality random number generator might be needed within the context of a shell script.
My favorite tool for this is "openssl rand" and I can find the manual page on rhel7 with "man rand."
Here are 8 bytes of randomness in base64:
$ openssl rand -base64 8
mGHxYcVOmA8=
$ printf %d\\n 0x$(openssl rand -hex 2)
21924
The shell's $RANDOM is somewhat less trustworthy than other sources here. It also does not appear to be POSIX, but comes from ksh88, where it is described as returning a random integer between 0 and 32767 each time it is referenced. If unset in a script, it cannot be used again for the life of the shell.
I use this to create passwords when I need to manually:
function password_generate { head -c ${1-12} /dev/urandom | base64; }While being overly pedantic (because "not true" random is a spectrum), it is good practice to use random numbers to ensure that a recurring cron job does not run at the same time on all machines.
This goes from good practice to necessary as a cron job uses increasing resources or talks to a centralized service.
Puppet implements this as splay: https://www.puppet.com/docs/puppet/7/configuration.html#spla...
Systemd offers multiple options for randomized delays for timers, documented at: https://www.freedesktop.org/software/systemd/man/systemd.tim...
You don't need the shell for that. Every modern cron supports randomization internally.
Set RD to $N random bytes in decimal:
RD=$(dc -e 16i`xxd -c $N -g $N -l $N -p /dev/urandom | tr a-f A-F`p)Unfortunately, this is neither readable nor particularly concise.
If I recall correctly, /dev/urandom and /dev/random work the same now.
That change was reverted, and I do not know if they were re-unified since.
https://lwn.net/Articles/889452/
Pragmatically, they are equivalent for most scripting use cases.
> As always, my advice for things like this is that if you’re in need of true random numbers in your shell script then you should probably stop writing whatever it is you are writing as a shell script.
The use of "true" here seems ambiguous.
Does this statement mean that you discourage creation of cryptographic applications with shell scripts, you discourage use of shell scripts to orchestrate applications that use /dev/urandom for any reason, or something else entirely?
(first, and only edit: This comment is in good faith, so if you're downvoting please respond to let me know why)
"True random" as opposed to "pseudo random", which is not cryptographically secure.
Are you on GNU/Bash version 5.1[0]? A new bash variable, SRANDOM added to GNU/Bash-5.1 release, which gets its random data from the system’s entropy engine and is not lined and cannot be reseeded to get an identical random sequence. For example:
echo "$SRANDOM"
for r in {1..5}; do printf "%s\n" "$SRANDOM"; done
Or something like this; echo -e "$SRANDOM"{1..5}"\n"
Did you really want to append digits 1, 2, 3, 4, 5 to your high-quality random numbers?
Shameless plug: my bc [1] has a random number generator that is much higher quality than the shell one. Because it's bc, you can generate random numbers of any size, and you can generate random reals as well.
It is also seeded.
Can you give an example usage? I cannot figure out how to invoke your PRNG.
If you want just a number to store in a hardware integer type, use this:
echo "rand()" | bc
echo "irand(<bound>)" | bc
It can also be used to generate either really small or really large integers, arbitrary size. My bc will take care of everything behind the scenes.
If you want to generate a real between 0 and 1, use this:
echo "frand(<places>)" | bc -l
If you want an arbitrary real (not between 0 and 1), use this:
echo "ifrand(<bound>, <places>)" | bc -l
echo "seed = $SEED; rand()" | bc
echo "seed" | bc
OUT=$(echo "rand(); seed" | bc)
RAND=$(echo "$OUT" | head -n1)
SEED=$(echo "$OUT" | tail -n1)
RAND2=$(echo "seed = $SEED; rand()" | bc)
This pattern works because the `seed` variable is updated after every call to the PRNG.
Does that help?
Fun fact: bash's $RANDOM never yields the same value twice in a row:
$ for i in {1..999999}; do echo $((RANDOM-RANDOM)); done | sort -nu | grep -E '^-?.$'
-9
-8
-7
-6
-5
-4
-3
-2
-1
1
2
3
4
5
6
7
8
9Maths isn't my background, but isn't this a serious flaw with the quality of the pseudo-random numbers it generates?
I remember learning about the enigma machine and how it had a flaw where a letter could not be transformed into itself. And knowing that, for example, an A could never be a A was critical to cracking the enigma code. Because of that knowledge, my cryptographic intuition is that any seemingly minor fact like this could actually be a critical flaw.
I doubt it matters much. Bash's $RANDOM is implemented using a linear congruential generator, which is not suitable for cryptographic applications anyway.
I use "openssl rand X" where X is the number of pseudo-random bytes i need.
Say you want to generate a strong password, you can use this handy function that works in bash and zsh:
function gen_passwd() {
openssl rand 240 |\
LC_CTYPE=C tr -dc '[:graph:]' |\
cut -c 1-${1:-20}
}If you have a password manager installed on your system (and you should) then it might have a command-line client that can generate new passwords. e.g.:
karellen@localhost:~$ keepassxc-cli generate -lU
hpJUwcYZnzJyAbrnvTWnTzrlqzqcdvsl
karellen@localhost:~$ keepassxc-cli diceware
diabetes dinginess goldfish purr cried expenses related
Or just use what you already have on your machine...
sort -R /usr/share/dict/words | head -n 4| sed 's/.*/&/;$!s/$// ' |tr '\n' '-' |sed 's/-$/\n/'Personally I have a tiny shell function that uses openssl rand with a dice ware list. It accepts a length, and it returns CamelCase.
I use it for usernames since Bitwarden doesn’t support diceware names.
Seems odd that these convenience function don't draw from either /dev/random or /dev/urandom/
I'm pretty sure $RANDOM predates /dev/{u,}random on most OSes.
From the conclusion of the OP:
> I could do more experiments and try to definitively determine what's happening, bottom line, don't use $RANDOM, even for unimportant random numbers.
As hexdump tends to always exist (vs python) I've used the following before:
hexdump -e '"%d"' -n2 /dev/urandomI used this to create random decisions for a while:
if (( RANDOM % 2 == 0 )); then echo Yes; else echo No; fi
bit=$(od -An -N1 -i /dev/urandom)
if (( $bit &1 ))
then echo "Yes"
else echo "No"
fiThere's always `head -c20 /dev/random | base64`
likely urandom but the point still stands.
Another fun fact: on Linux systems, the /proc/sys/kernel/random/uuid file contains a random UUID. This can be used to get 32-bit random numbers in POSIX shell without calling out to external utilities. Quite handy on OpenWrt.
read -r < /proc/sys/kernel/random/uuid
MYRAND=$(( 0x${REPLY%%-*} ))
You can also use "shuf" to create random listen port for aria2c.
PORT=$(shuf -i 1000-2000 -n1)
aria2c --listen-port=$PORT ...other...options...
The problem with random as a word is it covers user intents ranging from "crudely load balance well enough most of the time" or "add a small variable delay to smooth the load over time" up to extremely security critical crypto applications.
I assume that the random numbers in bash come from an uniform distribution. Is there a way of getting random numbers from a different distribution with given parameters?
I can only think of one-liners in R or maybe python.
The standard approach is to take the CDF form of the desired distribution, solve it for equaling some constant, then plug in your uniform value and out comes where on the desired distribution that maps to. The hard part is you either have to pre-solve it or use numerical solving methods and in either case you'd have to write that in Bash.
for a normal distribution, chatgpt suggests the box-muller transform and calling out to awk (which is a scripting language that continues to surprise me; it seems underrated?)
normal_random() {
start=$1
end=$2
range=$(echo "$end - $start" | bc -l)
awk -v start=$start -v range=$range -v seed=$RANDOM '
BEGIN {
srand(seed);
u1 = rand();
u2 = rand();
z0 = sqrt(-2 * log(u1)) * cos(2 * 3.14159265358979323846 * u2);
z1 = sqrt(-2 * log(u1)) * sin(2 * 3.14159265358979323846 * u2);
random_number = start + (z0 * (range / 6)) + (range / 2);
printf("%.0f\n", random_number);
}'
}
EDIT: Nope, there's definitely something wrong with this, possibly the rand() call(s)? Can anyone guess? (A real-world example of the perils of relying on chatGPT!) EDIT2: Figured it out. srand with no args seeds to the current second, making any run of this function within the same second produce the same result. Fixing this left as an exercise to the reader EDIT3: I fixed it by seeding it externally via $RANDOM.
On a related note, I've once had to reproducibly generate a random sequence of N-bytes, and one of the more elegant ways I discovered and ended up using was to 'encrypt' a known sequence of bytes with the required length (so that the sequence will be long enough) with some known 'password', and truncating it.
`( yes '00' | head -n 100000 | tr -d $'\n') | xxd -r -p | aespipe -e AES256 -P <(echo 'your-seed')`
Since no-one asked, I use something like this in zsh:
rand() {
local r4
IFS= read -rk4 -u0 r4 < /dev/urandom || return
local b1=$r4[1] b2=$r4[2] b3=$r4[3] b4=$r4[4]
print $(( #b1 << 24 | #b2 << 16 | #b3 << 8 | #b4 ))
}
Anyone want to critique this deterministic random number generator I wrote not long ago in Bash that takes advantage of sha256 sums?
https://gist.github.com/pmarreck/e65f457e8755e461a87db7c94d4...
$((32#$DRANDOM_SEED)) is not correct: the sha256 hash is in base 16, not 32. With your current implementation every fifth bit is always 0.