Today we’re going to discuss usenet, also known as newsgroups. Usenet is a great tool for finding interesting things on the internet. Before we get too deep into this tutorial, readers should note that all the information provided here is for educational purposes only.

Wikipedia defines usenet thusly:

Usenet is a worldwide distributed Internet discussion system. It was developed from the general purpose UUCP dial-up network architecture. Duke University graduate students Tom Truscott and Jim Ellis conceived the idea in 1979 and it was established in 1980.[1] Users read and post messages (called articles or posts, and collectively termed news) to one or more categories, known as newsgroups. Usenet resembles a bulletin board system (BBS) in many respects, and is the precursor to Internet forums that are widely used today. Usenet can be superficially regarded as a hybrid between email and web forums. Discussions are threaded, as with web forums and BBSes, though posts are stored on the server sequentially.

One notable difference between a BBS or web forum and Usenet is the absence of a central server and dedicated administrator. Usenet is distributed among a large, constantly changing conglomeration of servers that store and forward messages to one another in so-called news feeds. Individual users may read messages from and post messages to a local server operated by a commercial usenet provider, their Internet service provider, university, employer, or their own server.

I like to think that usenet works a lot like gossip does in life. Thinking about the files that users share on usenet as bits of gossip also helps me to understand the different roles the players in the usenet world have. Because I think it might be easier to understand, we’re going to use this analogy today.

Gossip in Real Life

In life people react to juicy bits of information in different ways. There are people who don’t partake at all and choose to forget gossip as soon as they hear it. Some use the information for themselves or might share it with a few other people who probably won’t share it with anyone. Then there are the kind of people who just can’t help but to tell everyone every juicy bit of information they get. The first person they usually run to is another person who loves to gossip. In this last scenario, a piece of information gets spread really quickly.

For the sake of our analogy, we’re going to use two people, Bobby and Lisa. Bobby and Lisa LOVE to gossip. In fact, anytime Bobby hears something, he immediately tells Lisa. Likewise, as soon as Lisa hears something, she immediately tells Bobby. Bobby has some friends that he talks to on a regular basis that he gets and shares gossip with and Susan has a different set of friends that she gets and shares gossip with. As you can see from this setup, even though Bobby’s friends and Lisa’s friends don’t talk to each other, because Bobby and Lisa share EVERYTHING, both of those groups have access to the same gossip.

Gossip on the Internets

Usenet works pretty much the same way as gossip does in life. Imagine the usenet servers are Bobby and Lisa. A user will have a local usenet server (Bobby) that they use to get and share information. That server then replicates what it finds out from its users to other usenet servers (Lisa), who in turn, share that new information with its users.

The advantage of this method is that if Lisa is hit by a bus, the gossip lives on. Likewise, if one usenet server goes down, others still have the content out there. Which is not to say that usenet doesn’t have it’s shortcomings. Just like in the real world of gossip where people tell things that aren’t true, users might upload faulty or incorrect things to usenet servers that then get replicated. They may name a file wrong, upload something of bad or horrible quality, or they might upload something malicious. When this happens, index sites (sites that index what’s available on usenet) mark this information as “nuked”.

Who is My Local Usenet Server?

Traditionally speaking, this is probably your internet service provider. As time goes on, this is becoming less and less common. Even if it was common, you probably wouldn’t want to use your ISP’s usenet access because they don’t manage their gossip very well. You’re better off finding a private usenet provider that charges somewhere in the neighborhood of $7 to $30 a month for unlimited access. When it comes to usenet servers you’re looking for a few different qualities. Below are the things you’re looking for.

Retention – This is how long the usenet server will keep the gossip on their servers. Most quality servers keep files for more than 1600 days.

Speed – This is how quickly and reliably the server will share that information with you. If you have a download of 100 mbps but the server can only send the files to you at 10 mbps, you’ll probably want to find a better server. A good usenet server will max out your connection.

Connections – These are the number of simultaneous connections you can have with the usenet server. More connections usually means more speed.

Transfer – The amount of data the usenet server allows you to download. I prefer to have an unlimited account that I pay monthly for, but you might decide you only need a block account.

Price / Support – You’ll want to look for a usenet provider that offers the mix of the above features that suits your needs at the price you’re looking for. You probably won’t need to use customer service at all, but they should respond promptly if you have a billing concern or something of that nature.

In the resource section of our site, we have a few usenet providers for you to consider along with our recommendations.

How Do I Find Gossip?

Just like in life, few people care to know gossip about every possible subject. You might only be interested in gossip about celebrities, a particular company, or some other sub-group. Usenet’s answer to this need is what we call a “newsgroup“. Rather than dump every piece of gossip into one giant bucket, they split things up by subject into specific newsgroups. Traditionally, you’d have to get a news reader program, install it on your computer, configure it to talk to your local usenet server, and then navigate to a specific newsgroup. You would then download what was known as the “headers” for the newsgroup. Think of “headers” like subjects for a bunch of emails. So if you were searching for gossip about downsizing at a company called Innotech, you might navigate to the newsgroup alt.innotech.rumors. You would download the headers and then do a search for “downsizing”.

Usenet and Attachments

Suppose we’re back in the early days of usenet and you’re discussing this downsizing on one of these newsgroups and you want to share a picture you have of security walking someone out, a juicy bit of gossip too good not to share. Just like an email, you would attach this picture to your discussion for others to download and view. Attachments were usually limited to 50 megabytes though. So if you had a large file to share, like a video of security walking them out, you zipped it up into a bunch of smaller files less than 50 megabytes each and then simply created a bunch of discussions each with a piece of the file and titled in a similar fashion. Users on usenet referred to these types of files as “binaries”. This took off, for obvious reasons, and soon there were newsgroups titled like, “alt.binaries.walkouts” and things of that nature. To get the video, people would do a search for what they wanted from the headers of a newsgroup and then download all the attachments from all the threads and then use their zip utility to combine and unzip all the attachments.

Even with better newsgroup readers, it was still a pain to search for what you were looking for. Suppose you were trying to find something you thought might be in, “alt.innotech.discussion”, went through the trouble of downloading all the headers and searching only to find out what you were looking for was actually in “alt.innotech.discussions” (note the “s” on the end). Users found this very frustrating until someone thought of .nzb files.

HOORAY! NZB Files!

NZB files are small xml files that contain information about where exactly to find a specific piece of data. Suppose you wanted to share with your friend the picture you uploaded of the person being escorted out by security. Rather than telling them to go to alt.innotech.rumors and searching for “downsizing” and finding your particular thread, you’d send them a .nzb file with all the information about where to find that file. Within the .nzb file it would say which newsgroup, which header, which thread, and which attachment to download. They fed that .nzb file to the newsgroup reader and it did all the work for them. The benefit being that they didn’t have to waste time downloading a bunch of headers, it would just go get the file they were looking for.

Usenet Index Sites

Several websites popped up that took this concept even further. They created a crawler that scours usenet on a regular basis looking for attachments and then indexes them on a site by creating a .nzb file for people to download. So now, rather that searching usenet directly for the gossip you’re looking for, you just go to the NZB index site, search for what you want there, and then download the associated .nzb file. You feed that file to your newsreader (SABnzbd) and it handles the dirty work of finding the newsgroup, the headers, the threads, and the attachment you want. Newsreaders like SABnzbd take it even further by automatically checking the attachments for errors, going to another usenet server to find missing attachments if your local usenet server doesn’t have them, repairing the attachments, combining the pieces, unzipping the file, moving that file to the directory you want, and the deleting all the left over files from the unarchiving process.

Putting it All Together

Now that we have index sites that neatly wrap things into small .nzb files we can feed to our news reader, and that newsreader automatically accesses our usenet to get the information and then process that information to our liking… usenet has truly become a fast, efficient, and secure way to get the information we’re looking for.

The Advantages of Usenet over Torrents

The only parties involved in a usenet download are you and the usenet provider. Peer-to-peer networks like torrents are essentially a big cloud of users connecting together and sharing pieces of the same file. On torrents, there’s nothing to stop anyone from jumping into that cloud and writing down IP addresses of other peers. Because any third-party can jump on the torrent and see who’s involved, what you download on a torrent is public for anyone to see.

By contrast, usenet is a one-to-one connection, just you and your provider. For added security and protection, usenet allows connection via SSL protocol. SSL protocol encrypts the information that’s sent back and forth between you and the usenet provider. So even if your ISP is doing something called “packet sniffing” where they try to figure out what you’re downloading by looking at the packets of information you’re sending, they’ll only see gibberish.

Another advantage of usenet is that you get full speed downloads from start to finish. A good usenet server will max out your connection. No more waiting for days because one torrent has one seeder and 3,000 leechers. Speaking of seeder to leecher ratio, there’s no concern for that either. Download as much as you want without worrying about uploading anything or maintaining a seed ratio to enable downloads.

Usenet Series: Next Step

If you plan on following my usenet series, you should go to the resources section of the website next and have a look at our list of usenet providers. There you’ll find a quick overview of selecting a usenet provider and our recommendations. You should also look at our section on NZB sites. You’ll want to find one you can get access to, sign up, and probably have at least one site with an upgraded or VIP account. Before you go to the next section about SABnzbd, you should have a usenet account with one of the providers we have listed and access to at least one NZB index site.