Hacker Public Radio

This show has been flagged as Clean by the host.

This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems.


Thirteen is one of my favorite numbers, so I am going to use this episode to indulge myself. As part of UNIX Curio #8 (
HPR episode 4657
), I went on a little flight of fancy and suggested that the
comm
utility could be used in place of
cat
. On further reflection, I realized that there are quite a few utilities that can do that job, almost. This episode is the result of me investigating how many possible ways there are to replace that functionality with standard utilities. Imagine that some prankster has deleted the
cat
utility from your machine, and you need a way to restore that capability without re-installing it.



I should warn you that as opposed to the rest of the series, this episode will not focus heavily on obscure tools, and it is unlikely the examples given will be very useful. Also, I am not going to dive into the history of any of these commands. However, you might still learn something, and it could be a little entertaining to hear about the ways some utilities can be twisted to this purpose.



Let's start by reviewing what the
functionality of




cat


[1] is. Under POSIX, all it does is read the data provided on standard input or from each file named as an argument, and writes it to standard output. Briefly, it con
cat
enates all of the entries. The only option required by the standard is
-u
, which causes
cat
to write data without any delay (if that option were not given, the program could choose to buffer the data). While many implementations offer additional options that modify the output, I am going to completely ignore them and just focus on that one job. The files named as arguments are processed in the order in which they are named. One or more of these can be
-
, in which case
cat
will use the data provided on standard input and then continue to the next file in the argument list. I will actually narrow things even further and allow discussion of cases where only one file is involved, or where
cat
is reading from standard input via redirection from a file.



I will start off with the
TL;DR
conclusion—none of the standard utilities are able to fully duplicate its functionality on their own. Some of them come close, but can't quite be relied on in all respects. One important reason for this lies with the
POSIX definition of a text file
[2]. This is a file treated as being separated into lines, divided by newline characters. There are three limitations: none of the lines can contain a NUL byte (value 0), the lines cannot exceed the maximum line length defined by the system (required to be at least 4096 bytes including the trailing newline), and the file must end with a newline character. (The last rule is implied by the
definition of "line"
, which states a newline must appear at the end of a line.) It might seem strange, but a text file
may
contain non-ASCII data; the only restriction on the values of bytes is that they cannot be NUL. Many of the standard utilities are only required to operate on text files and therefore cannot necessarily be relied on to handle arbitrary data. While a particular implementation might not have these limitations, there is no guarantee of that. I should note that both
cut
and
paste
are only required by POSIX to operate on text files, but with the exception that the lines can be of any length.



In addition to
cat
, the only utilities covered in this episode that are required by POSIX to handle any arbitrary file are
dd
,
tail
(when used with the
-c
option),
tee
,
tr
, and
uuencode
. The wording for the
compress
utility does not specifically
say
it must handle any file, but in practice I think this is true. However, all of these utilities have at least one limitation of their own: they cannot handle more than one file or they can only operate on standard input. Some other utilities are able to deal with multiple files, standard input, or both, but those are only required to operate on text files. In practice, this typically means that if an input file does not end with a newline character, the program will add one to the output, destroying the idea that the output of
cat
should be identical to the input.



Let's start with the least-capable options and work our way towards those that could serve as closer drop-in replacements. The first candidate is a utility that I thought might be persuaded to work, but it doesn't quite. This is


nl




, a command used to number lines of files or standard input
[3]. It normally prints the content of this input and, on the left, prints the number of each line. It
is
possible to turn off the numbering, and to set the separator between the numbers and the text to a null string. However, it does not seem to be possible to have absolutely nothing appear in front of each line. The POSIX specification states that the
-w
option can be given to define how many characters to the left of each line will be reserved for the line numbers—while the default value is six, it says nothing about what values must be accepted. I tested several implementations (specifically, from GNU, FreeBSD, NetBSD, and OpenIndiana, which is an open-source derivative of System V) and none of them allow setting the width of the space for line numbers to be less than one. Even if it were possible to remove the preceding space entirely, POSIX only requires
nl
to accept one file argument and to operate on text files, so it would still be limited.





In the examples for this episode,


myfile


is used to represent the name of the file being operated on. The text


myfile ...


means a filename possibly followed by other filenames (with no limitation on the number). The




nl




example below prints a single space in front of each line.




nl -b n -s "" -w 1 myfile



Next up are methods that actually work, but are only usable with standard input. When combined with input redirection from a file, they can accept a single file and send it to standard output.
The




read




utility
[4] takes a line of input and assigns it to a shell variable. It requires a little help, however—it can't output what it reads, so it has to be paired with something like
echo
or
printf
. Because it reads input one line at a time, it needs to be used in a loop to read the entirety of the data. In addition, the
IFS
shell variable must be set to a null string to prevent
read
from doing field splitting; we don't want it to modify the data being read at all. The shell loop
while IFS="" read -r foo; do echo "$foo"; done < myfile
will do the job, except that
read
is only required to handle text files. It is not clear to me whether POSIX allows the shell to have a limit on how much data can be stuffed into a variable; if it does, that could be a problem even if your implementation of
read
can handle lines longer than the maximum length.





The




tr




utility
[5] is another one that only works with standard input. It takes the input, replaces any characters in the first string argument with the corresponding characters in the second string argument, and sends the result to standard output. If the first and second strings are the same, such as
tr a a < myfile
, it will behave like
cat
.





The




tee




command
[6] also fits into this category. It copies standard input to standard output, just like
cat
does. It also sends a copy to any filename arguments given. If none of these are present, it just passes the data through untouched. You could use
tee < myfile
to print a file. Be careful, however, not to enter your filename as an argument (as in
tee myfile
), as that will cause it to be overwritten! The good news with both
tr
and
tee
is that they aren't limited to working with text files.



Let's now consider a couple of options that only work on a file, not with standard input. In addition, they can only handle
one single
file. This might seem to defeat the purpose of
cat
, which can combine multiple files, but
cat
is often used with only one file. These are


ed


[7] and


ex


[8], two text editors. By feeding them the right command sequence on standard input, they can be convinced to output the entire file and then exit. The appropriate invocations would be
printf '%s\n' '1,$p' Q | ed -s myfile
and
printf '%s\n' '1,$p' 'q!' | ex -s myfile
. The POSIX specification for
ex
indicates that more than one filename can be given as an argument. I attempted to take advantage of this with the command line
while true ; do printf '%s\n' '1,$p' 'n!' ; done | ex -s myfile ...
, which should print the first file named as an argument to standard output and move to the next one. In my reading of the specification, once the last file has been printed, the
n!
command (meaning "next") should result in an error, causing the program to exit. However, the implementations that I tested don't exit and just stay on the last file, printing it out endlessly. Perhaps I'm misunderstanding how
ex
works, but regardless, both it and
ed
are only required to handle text files, so can't completely serve in place of
cat
anyway.



The following category of replacements is capable of using either standard input or a file, but they can only work with one file. Let's start with


comm


[9], which was responsible for me exploring this question. The command
comm myfile /dev/null
will send
myfile
to standard output. If you want to use standard input instead of a file, you need to replace the first filename with a hyphen (as in
comm - /dev/null < myfile
). The


paste




utility
[10] works similarly. It can only be used with one file because if more are given, it will try to interleave the lines of these files together. Another possibility is


tail


[11]. When used with the
-c +1
option, it outputs the entirety of the file. Unlike other implementations I tried, the NetBSD version of
tail

is
capable of taking multiple filenames as arguments. However, in that case, it precedes the content of each file with the filename like
head
does, so doesn't produce a clean copy.
The




sort




utility
[12] used with the
-m
option, which I talked about in UNIX Curio #11 (
HPR 4687
), can also do the job. All of these are only required to handle text files by POSIX except
tail
, which must handle any arbitrary file when the
-c
option is used.



One more utility that can take either one file or standard input is


dd


[13]. It is a bit different from these others in a couple ways. First, it writes statistics about the data processed to standard error, so you would need to redirect that to
/dev/null
to replicate the behavior of
cat
. To use standard input, no arguments are needed. The syntax for arguments is unusual compared to other UNIX utilities; for this purpose,
dd if=myfile 2>/dev/null
would be the command to use.



The


compress


[14] and


uuencode


[15] utilities are in the same category, but do a lot more processing on the data along the way. The purpose of these is described respectively in UNIX Curios #7 (
HPR 4647
) and #1 (
HPR 4587
). Their output can be piped into a corresponding
uncompress
or
uudecode
utility to get back the original data and send it to standard output. The two commands would be
compress -c myfile | uncompress -c
and
uuencode myfile /dev/stdout | uudecode
—if you wanted to use standard input instead, just omit the filename arguments. I should point out that when I tested the
uuencode
method on FreeBSD 15.0, it didn't work; however, it did work on NetBSD 10.1 and OpenIndiana 2025.10.



Next, we have two possibilities that come very close, but don't quite behave
exactly
like
cat
. They send to standard output either the files named as arguments in order, or, if there are none, standard input. However, POSIX does not require them to take a hyphen as an argument, representing standard input. First up, we have


sed


[16]; when called with an empty script (as in
sed "" myfile ...
or
sed :a myfile ...
), it just prints out the contents without any transformations.
The




grep




utility
[17] works the same way—when given an empty regular expression (as in
grep "" myfile ...
), it treats that as matching anything and prints each line of input to standard output.



Finally, we have arrived at the utilities that can fully replace
cat
, albeit only for text files. They can be given one or more files, including a hyphen representing standard input, and will output them in the order they are given as arguments. If no filename arguments are given, they just use standard input instead. Starting out, we have


awk


[18]. While an empty pattern matches every line of input and an empty action prints the entire line, POSIX states that if both the pattern
and
action are empty, the utility is to simply exit without reading or writing anything. Therefore, we need to at least provide a pattern or an action—a pattern, as in
awk "/.*/" myfile ...
, can be shorter. Another one that has the capabilities is


cut -b 1- myfile ...


[19]. However, as noted earlier, these will add a newline character if the input doesn't already end with one, so they aren't exact replacements.



I was hoping that a simple invocation of one of the standard utilities could be used in place of the
cat
program that our prankster has deleted; however, I think I've demonstrated that isn't possible. Let's return to those utilities that can handle arbitrary files, but can't take more than one at a time. It
would
be possible to write a shell script to manage any options and arguments, then use the utility to do the actual work of sending through the data file by file. In my view,
tail
would be the best fit because it can take a bare filename as an argument and, unlike
dd
, uses the standard syntax for UNIX utilities.



The following script does just that—unfortunately, it is not really something simple that could be typed in from memory. So probably it would be easier in most situations to just re-install
cat
on your system instead of looking up this script, typing it in, and saving it where
cat
used to be.



#!/bin/sh
# Exit if any option other than -u is given; 'getopts' will issue an error
# message. We don't actually do anything different if -u is given.
while getopts u catopt
do
if [ "X$catopt" != "Xu" ]
then
exit 1
fi
done

# Reset the positional parameters to remove any options given.
shift $(($OPTIND - 1))

case $# in
0)
tail -c +1
;;
*)
for fname
do
case "$fname" in
-)
# Single hyphen represents standard input, same as with no arguments.
tail -c +1
;;
*)
# The -- ensures that $fname will be treated by 'tail' as an argument,
# not an option.
tail -c +1 -- "$fname"
esac
done
esac



It might be tempting to implement this substitute as an alias or a shell function rather than as a script, but those only exist in the current shell execution environment (and its subshells) and are not inherited by utilities and scripts run by you or others. So the easiest way to get it to be used by other scripts such as
cron
jobs would be to save a copy of it on your system where
cat
normally lives, making sure that
the "execute" file mode bits are set
.



There are certainly other widespread tools, such as Perl, that could also be used in place of
cat
. However, I wanted to focus only on utilities that are so ubiquitous that they appear in the POSIX standard. It should be noted that even some of these, such as
compress
and
uuencode
, aren't always present on modern systems.



References:







Cat specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cat.html





Definitions: Text File
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403





Nl specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/nl.html





Read specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html





Tr specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/tr.html





Tee specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/tee.html





Ed specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ed.html





Ex specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ex.html





Comm specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/comm.html





Paste specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/paste.html





Tail specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/tail.html





Sort specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html





Dd specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html





Compress specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/compress.html





Uuencode specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/uuencode.html





Sed specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html





Grep specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html





Awk specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html





Cut specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cut.html







Provide feedback on this episode.

This show has been flagged as Clean by the host.

Python and its libraries

1. librosa — loading and resampling audio

def load_audio(path):
y, sr = librosa.load(path, sr=SR, mono=True)
return y, sr

Every episode, regardless of its original sample rate or channel count, gets loaded down to a single mono stream at 16kHz (SR = 16000). That's one line of librosa, resampling, and folding stereo to mono. It made every downstream tool (VAD, pitch tracking, MFCC extraction) operate on a consistent, predictable input regardless of what an MP3 encoder or podcast host originally used.

2. parselmouth (Praat bindings) — pitch and voice-quality tracking

def pitch_and_hnr(y, sr):
snd = parselmouth.Sound(y, sampling_frequency=sr)
pitch = snd.to_pitch(time_step=PITCH_TIME_STEP, pitch_floor=PITCH_FLOOR, pitch_ceiling=PITCH_CEILING)
f0 = pitch.selected_array["frequency"]
times = pitch.xs()
try:
harmonicity = snd.to_harmonicity_cc(time_step=PITCH_TIME_STEP)
hnr = np.array([harmonicity.get_value(t) for t in times])
hnr = np.nan_to_num(hnr, nan=-999.0)
exceptException:
hnr = np.full_like(f0, 999.0) # fail open if HNR extraction errors
return f0, hnr, times

parselmouth is a Python wrapper around Praat, the standard tool in phonetics research. This is the heart of the whole vocal-monotony measurement: to_pitch() gives fundamental frequency (F0, roughly "pitch") every 20ms, and to_harmonicity_cc() gives harmonics-to-noise ratio (HNR) at the same timestamps — a signal-quality score used later to throw out pitch estimates taken from noisy or badly-compressed audio rather than let bad audio masquerade as "flat" pitch.

3. webrtcvad — deciding which frames are actually speech

def vad_speech_mask(y, sr, aggressiveness=2):
vad = webrtcvad.Vad(aggressiveness)
pcm16 = (np.clip(y, -1.0, 1.0) *32767).astype(np.int16)
n_frames =len(pcm16) // VAD_FRAME_LEN
mask = np.zeros(n_frames, dtype=bool)
for i inrange(n_frames):
chunk = pcm16[i * VAD_FRAME_LEN:(i +1) * VAD_FRAME_LEN]
raw = struct.pack("%dh"%len(chunk), *chunk)
try:
mask[i] = vad.is_speech(raw, sr)
exceptException:
mask[i] =False
return mask, times

webrtcvad is the voice-activity detector out of Google's WebRTC project — a frame-by-frame classifier ("is this 30ms chunk speech or not"). It needs raw 16-bit PCM bytes, not floats, hence the np.clip and struct.pack conversion.

4. librosa.feature — telling music apart from speech

flatness = librosa.feature.spectral_flatness(y=y, hop_length=hop)[0]
chroma = librosa.feature.chroma_stft(y=y, sr=sr, hop_length=hop)
onset_env = librosa.onset.onset_strength(y=y, sr=sr, hop_length=hop)
...
chroma_diff = np.mean(np.linalg.norm(np.diff(chroma_win, axis=1), axis=0))
chroma_stability =1.0/ (1.0+ chroma_diff)
...
scores[w] =0.4* chroma_stability +0.35* periodicity +0.25*min(flat_mean *3, 1.0)

Podcast intros/outros are often music, and music has a very different pitch profile than speech — including it would bias the "monotone" measurement. Rather than train a classifier, three librosa features were combined with hand-tuned weights: spectral flatness (music tends to have more tonal structure, less noise-like spectrum), chroma stability (sustained harmony vs.

constantly-changing speech formants), and onset periodicity (a regular beat vs. speech's irregular rhythm).

5. librosa.feature.mfcc + sklearn.cluster.KMeans + sklearn.metrics.silhouette_score — an unsupervised speaker-count attempt

mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13, hop_length=hop)
...
chunk_vecs.append(np.concatenate([seg.mean(axis=1), seg.std(axis=1)]))
...
X = np.array(chunk_vecs)
X = (X - X.mean(axis=0)) / (X.std(axis=0) +1e-9)

best_k, best_score =1, -1.0
for k inrange(2, min(MAX_SPEAKERS, len(X) -1) +1):
labels = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
iflen(set(labels)) <2:
continue
score = silhouette_score(X, labels)
if score > best_score:
best_k, best_score = k, score

estimated = best_k if best_score >= SILHOUETTE_MIN else1

MFCCs (Mel-frequency cepstral coefficients) summarize the shape of a short audio window's spectrum — a rough "voice fingerprint" widely used in speech tasks. The idea: chop each episode into 2-second chunks, turn each into a fingerprint vector, standardize (z-score) the features, then use KMeans to see if the chunks naturally split into 2+ clusters (different voices) rather than 1 (a single voice throughout). silhouette_score scores how well-separated the clusters are, and picks the best k — or falls back to "1 speaker" if no clustering clears a minimum quality bar. This part of the investigation did not work, as it picked up noise as another host.

6. scipy.stats.mannwhitneyu + a hand-written Cliff's delta — comparing two groups without assuming a normal distribution

def cliffs_delta(a, b):
a, b = np.asarray(a), np.asarray(b)
gt =sum((x > y) for x in a for y in b)
lt =sum((x < y) for x in a for y in b)
return (gt - lt) / (len(a) *len(b))


def compare_group(df, metric, label):
hpr = df[df.corpus =="hpr"][metric].dropna()
cmp= df[df.corpus =="comparison"][metric].dropna()
u, p = stats.mannwhitneyu(hpr, cmp, alternative="two-sided")
delta = cliffs_delta(hpr.values, cmp.values) iflen(hpr) *len(cmp) <200000elseNone

scipy.stats.mannwhitneyu is a statistical test: it compares two groups of numbers (HPR vs. comparison podcasts, per metric) without assuming they're normally distributed — appropriate for skewed, real-world data like pitch-range measurements. scipy doesn't ship an effect-size measure alongside it, so Cliff's delta — "of all possible (HPR episode, comparison episode) pairs, how much more often is HPR's value the smaller one?" This pairing (a significance test plus an effect-size number) was used throughout because a low p-value alone doesn't say whether the difference is actually big enough to matter.

7. pandas — grouping, stratifying, and cross-tabulating results

print(df.groupby("corpus")["retained_fraction"].describe())
...
print(pd.crosstab(df["corpus"], df["register"]))

Once data/results.jsonl (one JSON object per analyzed episode) is loaded into a pandas.DataFrame, two one-liners answer two different questions that would otherwise need custom code: .groupby(...).describe() gives a distribution summary (count, mean, std, quartiles) of audio-quality coverage per corpus, in one call, and pd.crosstab() cross-tabulates HPR vs. comparison against male-typical/female-typical register — a quick sanity check that the two collections of podcasts had a similar gender-register mix.

8. matplotlib — box plots for the headline comparison

fig, ax = plt.subplots(figsize=(6, 4))
data = [df[(df.corpus == c) & df[metric].notna()][metric].values for c in ["hpr", "comparison"]]
ax.boxplot(data, tick_labels=["HPR", "Comparison"], showmeans=True)

A box plot per pitch-range metric, HPR next to comparison podcasts, mean marked as well as the median.

9. spacy word vectors — measuring topic drift, not just word choice

nlp = get_nlp() # spacy.load("en_core_web_md")
...
chunk_docs = [nlp(c) for c in chunks]
chunk_vecs = []
for d in chunk_docs:
v = d.vector
norm = np.linalg.norm(v)
if norm >1e-6:
chunk_vecs.append(v / norm)
...
adj_sims = [float(np.dot(V[i], V[i +1])) for i inrange(n -1)]

Each ~130-word chunk of a transcript is embedded as a single vector using spaCy's medium English model (en_core_web_md, which has built-in word vectors), then L2-normalized so a dot product between two vectors
is
their cosine similarity. Comparing consecutive chunks' similarity gives a "does the topic barely move between one chunk and the next" score.

10. lexicalrichness — vocabulary diversity that doesn't cheat on transcript length

lr = LexicalRichness(text)
try:
mtld = lr.mtld(threshold=0.72)
exceptException:
mtld =None

A naive vocabulary-diversity score (unique words ÷ total words) gets worse the longer a text runs, because of how the metric is defined — not because the speaker is actually repeating themselves more.

MTLD (Measure of Textual Lexical Diversity) avoids that by sweeping through the text and counting how many words it takes before diversity drops below a threshold, then averaging — a metric designed specifically to stay comparable between a 10-minute and a 40-minute episode.

11. vaderSentiment — a lexicon-based sentiment score

vader = get_vader() # SentimentIntensityAnalyzer()
sentiments = [vader.polarity_scores(c)["compound"] for c in chunks]
...
"sentiment_std": float(np.std(sentiments)) iflen(sentiments) >1elseNone,

VADER scores each chunk's sentiment from -1 to +1 using a hand-built lexicon and rule set (not a neural network) — chosen for being fast, deterministic, and cheap enough to run per-chunk across hundreds of episodes. The value then used isn't the average sentiment — it's the
standard deviation
of sentiment across an episode's chunks: does the emotional

tone ever move, or does it stay flat throughout?

12. zlib (standard library) — compression ratio as a gauge for entropy

def compression_ratio(text):
raw = text.encode("utf-8")
iflen(raw) <200:
returnNone
compressed = zlib.compress(raw, level=9)
returnlen(compressed) /len(raw)

No third-party NLP library involved at all — just the standard-library zlib module. A gzip-style compressor finds and exploits repeated patterns; if it can squash a transcript down a lot, that transcript was relatively repetitive and predictable, and if it can barely compress it, the wording was more varied. This is roughly tells you "verbal monotony."

13. re (standard library) — turning "does HPR favour solo hosts" into a checkable heuristic

MULTI_PRESENTER_PATTERNS = [
r"\bco-?host(?:s|ed|ing)?\b",
r"\bguest\b",
r"\binterview(?:s|ed|ing)?\b",
r"\btalks? with\b",
r"\bjoins? me\b",
...
]
CORRESPONDENT_LINK_RE = re.compile(r'correspondents/(\d+)\.html')

def multi_presenter_score(summary, notes):
text = (summary or"") +" "+ (notes or"")
hits = [p for p in MULTI_PRESENTER_PATTERNS if re.search(p, text, re.IGNORECASE)]
summary_links =set(CORRESPONDENT_LINK_RE.findall(summary or""))
return {
"flagged_multi_presenter": bool(hits) orlen(summary_links) >1,
}

Rather than the acoustic speaker-clustering attempt (item 5, which turned out to be unreliable), a more trustworthy stat came from plain text: HPR's own episode metadata (pulled from a local knowledge-base repo) has host-written summaries, and a short list of regex patterns ("guest", "interview", "joins me", etc.) plus counting linked correspondent profiles was enough to flag likely multi-presenter episodes.

14. multiprocessing.Pool — running the same analysis over hundreds of files without waiting hundreds of times over

work = [(r, local_paths[r["id"]]) for r in batch]
with mp.Pool(args.workers) as pool, open(args.out, "a") as out_f:
for result in pool.imap_unordered(_worker, work):
out_f.write(json.dumps(result) +"\n")
out_f.flush()

Every batch pipeline (acoustic pitch analysis, speaker-count pass) used the same shape: Pool(workers).imap_unordered(...) to run the per-episode function across multiple CPU cores at once, streaming results out to a JSON-lines file as each one finishes rather than waiting for the whole batch. imap_unordered was chosen over map so a single slow episode couldn't hold up writing out the results that had already finished.

15. subprocess + a resumable batch loop — working within a disk-space limit

def load_done_ids(results_path):
ifnot os.path.exists(results_path):
returnset()
done =set()
withopen(results_path) as f:
for line in f:
...
done.add(json.loads(line)["id"])
return done

...
rows = [r for r in rows if r["id"] notin done]
...
for batch_start inrange(0, len(rows), args.batch_size):
batch = rows[batch_start:batch_start + args.batch_size]
local_paths = fetch_batch(batch, args.scratch) # scp from the remote host
...
for p in local_paths.values():
os.remove(p) # delete before next batch

All the podcasts (~220GB) didn't fit on local disk (~54GB free). The fix was a loop: scp (via subprocess) one batch of audio in, analyze it, delete it, move to the next batch. load_done_ids() makes the whole pipeline resumable by reading back what's already in the output file, so a crash or a deliberate stop partway doesn't lose completed stuff.

Links

https://news.ycombinator.com/item?id=49110547

https://mastodon.social/@ngate/117009495478432200

Ah yes, the riveting world of Hacker Public Radio: a quaint little corner where tech wizards ✨️ divulge their magical secrets ✨️ to each other in the most riveting monotone known to man. Who knew endless "community" podcasts could be so... utterly riveting? 📻️🎙️ Apparently, everyone is just dying to contribute to this cacophony once a year—what a privilege! 😂

https://hackerpublicradio.org/

#HackerPublicRadio #techCommunity #podcasting #MonotoneMagic #rivetingContent #HackerNews #ngated


Provide feedback on this episode.

This show has been flagged as Clean by the host.












BruCON: https://brucon.org





BruCON
is an annual security and hacker conference providing two days
of an interesting atmosphere for open discussions of critical
infosec issues, privacy, information technology and its
cultural/technical implications on society. Organized in
Belgium, BruCON offers a high quality line up of speakers,
security challenges and interesting workshops. BruCON is a
conference by and for the security and hacker community.

​The conference tries to
create bridges between the various actors active in computer
security world, included but not limited to hackers, security
professionals, security communities, non-profit organizations,
CERTs, students, law enforcement agencies, etc…..




https://www.youtube.com/@brucontalks/videos




Provide feedback on this episode.

This show has been flagged as Clean by the host.

In this episode we look at a very old style of beer known as Mild. Paul, in the absence of any locally available Milds has a Guinness, Dave tastes
Banks's Mild
and Kevie Samples
1950s Burton Ruby Mild
.







Connect with the guys on
Untappd
:







Dave






Paul






Kevie




















The intro sounds for the show are used from:







https://freesound.org/people/mixtus/sounds/329806/






https://freesound.org/people/j1987/sounds/123003/






https://freesound.org/people/greatsoundstube/sounds/628437/










The next 3 beer styles to be reviewed:









Bitter



Pale Ale



Trappist







Provide feedback on this episode.

This show has been flagged as Clean by the host.

wl-copy is a command in the wl-clipboard package, and allows you to copy and paste data from your Linux Terminal on a Wayland desktop.

Copy an arbitrary string to your clipboard using wl-copy:


$ wl-copy "hello world"
Paste it anywhere on your system as usual (with Ctrl+Shift+V or a right-click and Paste).

You can copy from output, too:


$ echo "hello wide world" | wl-copy
You can redirect to wl-copy:


$ echo "hello world" > example.txt

$ wl-copy < example.txt
You can use wl-paste to paste from your system clipboard:


$ wl-paste

"hello world"
Provide feedback on this episode.