Intro to linux
I started posting on minds.com because I missed SysAdmin Magazine. I missed reading it, and I missed writing for it. It was such a good magazine that I bought back-issues on CD and then set them up on a web-server so I could search them all. They were articles written by System Admins, for System Admins and the signal to noise ratio was excellent. (The advertisers they attracted were usually pretty good too) And I figured I could start scratching that technical writing itch by the occasional post to minds.com
But it's been mentioned that I should also make some blog posts for people new to linux too. Not a bad idea! So I figured I'd start off some tips on how to get started with it, and a few cautions, and a brief, informal, not-fact-checked history to partly explain why linux works the way it does.
So, to get started, I always recommend someone take some old computer they're no longer using, download a “live CD” from one of the linux “distros” (slang for “distribution”), and boot it on their old system. Play with it a bit. See what works or doesn't work out-of-the box. This is especially important if you're trying it out on a laptop. Laptops often use funky chip-sets for WiFi and ethernet and graphics interfaces, which have sometimes not been 100% supported on linux. If it looks like most things you feel are critical are working, install it.
These days, linux systems have such easy to use window managers that any windows or mac user can figure it out, eventually. Heck, if my Father, in his 80s, can figure out how to use Ubuntu's Unity user interface, you certainly can! And that's one of the nicer things about linux. If you don't like something, odds are you can change it. My graphical console days started off with SunOS and .. er… I'm ashamed to admit I can't remember the name of the window manager it used now. :-) It had a NeWS interpreter built into it which I used at the time to make some pretty cool GUIs (graphical user interfaces – NeWS wanted to be what Java eventually was). So in my case, what I wanted for a graphical desktop was pretty basic and I never really cared much for Unity. But I didn't need to. On Solaris, 15 years ago, I ran a window manager that looked like windows 95 for a while. Then I switched to “enlightenment” which was developed on linux.
Currently, I prefer linuxmint with the cinnamon window manager. But that's just personal preference. Feel free to download several different Live CDs. Most of the linux distros now offer a “live CD” ISO file which you can burn to a DVD or USB stick. The idea is that you can boot linux right off of it and have a chance to play with it before you commit to installing it on the hard drive. So you could download linuxmint with the cinnamon, Mate, Xfce, or KDE window managers. They're all linux, they just have a different GUI interface, a different desktop.
And there are others. I like linuxmint because I've had good luck with it even on newish systems with newish chip-sets. It happens to be based on the ubuntu distribution which in turn is based on the debian distribution. And on really weirdo hardware, I've preferred debian, mostly because they seemed the best at making it easy to install/run on really odd hardware. For instance, my personal email/web server used to run on a DEC Alpha which I rescued from the recycling bin at work. (grin) And at one point someone else was throwing out some other, older DEC Alpha systems that were old and not terribly fast, but these cute little (approx) 8x8x3 inch boxes. So I slapped debian on them and started using them to learn about clustering software and programming. Yeah. I'm a geek. I admit it.
There are others too. CentOS and Fedora are both based on the commercial RedHat distribution. And then there's SUSE, Mandrake, the list just goes on 'n on.
But I recommend these Live CDs to kick the tires, test out how compatible it'll be on your hand-me-down hardware, etc. Then install it. Every distro will likely have slightly different tools for downloading and installing other software, so I can't offer pointy-clicky instructions here for all of them. But let google be your guide. There's tons of youtube videos out there as well as web forums with people asking whatever question you have and getting it answered. On the debian/ubuntu/mint flavors of linux, look in the admin tools menu for “Software Manager” or “Synaptic Package Manager”.
So now some cautions. Much like Unix in ye olden days, much of the opensource software is developed by geeks for geeks and sometimes there's a steep learning curve. Sometimes the documentation is not great or even missing (it's far more fun to write programs than to write documentation, believe me). But you'll find lots of online resources to help you out. And the good news is that a lot of the opensource software you'll find is often even more powerful than some commercial apps for windows… Once you figure them out.
As good as StarOffice/OpenOffice/LibreOffice/whatever-its-called-today is, it's not 100% compatible with MS Office. It's just not. It's mostly compatible, most of the time. However, at least the browser wars of the 90s are mostly over. You can browse pretty much any site with firefox, chrome, or chromium and have it look decent and work. I suppose people are now demanding MSIE be more compatible with their websites instead of websites demanding your browser be identical to MSIE, thank goodness.
Linux has been often accused of being arcane or “only for geeks or programmers” or “only for servers”. Well, the command line tools certainly can be arcane. But it's not like that to make it more difficult. The commands tend to be very generalized so you can use them in a jillion different ways for a jillion different circumstances. Why? Well……..
In ye olden days of computing, all the hardware vendors had their own operating systems. So if you wrote a program intended to run on it, getting it to run on some other computer with some other operating system was non-trivial. And they may not even have compilers for the same language you used to write the program or if they did, it might not be very portable to that vendor's compilers for their system. This was before the days of languages like C where you at least stood a halfway decent chance that your program would compile and run the same way on multiple systems.
Along came Unix. I believe it was developed at one or more universities and at AT&T Bell Labs. And it had a lot of cool ideas (more on that in a bit). And eventually, customers started asking hardware vendors like IBM or Digital Equipment Corporation if they could run Unix on it. Eventually, the hardware vendors even started making their own flavors of Unix to go with their hardware. So we started seeing SunOS (later Solaris), DEC Unix, HPUX, IRIX, BSD, NCR (National Cash Register) and AT&T both had their own flavors of Unix… It was on big servers and on workstations but both were pretty pricey.
One of the cooler things (I think it was first seen on Unix) was conceived at Bell Labs – the pipe. They had lots of different people making small, but useful programs for lots of different things. And someone (sorry, I'm not sure of names here) thought wouldn't it be cool if we could have all these smaller programs working together on a larger problem. And the pipe was born.
The idea is that you can have one program that reads input, does something rudimentary, then sends the output in such a way that it can be “piped” as input to another program, which does something further with the output of the first program, then generates it's own output. This sounds odd at first but it made it possible to quickly do some pretty impressive things with the rapidly growing collection of these tools.
The first example that springs to my mind (probably because I'm an IT geek these days) is wading through long log files and picking out just the bits you want. For instance, say I have a log file showing who logged into a system when and some of it's logs look like:
Jul 27 20:34:16 myserver sshd[2925]: Accepted password for fflintstone from 192.168.1.10 port 52283 ssh2
But there's a pile of other stuff in this log file, and all I want is to summarize what usernames logged in and how many times they logged in. Well, unix/linux to the rescue! I can just do
cat /var/log/auth.log |grep “Accepted password for” |sed 's/.*password for //' |awk '{print $1}] |sort |uniq -c |sort -r -n >users.txt
Ok, yeah, that looks horrifying, huh. But really it's just a collection of a bunch of different commands. Each one is pretty simple. The cat command just reads a named file (the auth.log file here) and dumps it to “standard out” - the output channel I mentioned. I pipe that with the | character as input to the grep command. The grep command uses “regular expressions” to find only certain lines and print out only the ones that the regex pattern matches, in this case lines that contain “Accepted password for”. That output is passed to the sed command which uses expressions to modify the input – in this case it erases/omits any characters up to and including “password for “ and then prints the remainder out. So the output lines it prints out will all start with the usernames in the logs. That output I send to awk just because it has an easy way to separate things (using spaces in this example) and it prints out only the first “word” it finds which in this case will be the usernames like fflintstone. That output I send to sort, then that sorted output I send to the uniq command which will count all the unique lines and print out the number it found and each line. So the output might look like:
25 fflintstone
12 brubble
etc…
Still sounds arcane, huh. But an IT geek who has to dig through logs all the time quickly gets to know several of these commands and some of the ways they can be used to do different things. And when you're working on hundreds of systems with tens of thousands of users, just using a spreadsheet and cutting and pasting millions of log lines into excel just doesn't scale well. And I can quickly take care of all sorts of ad-hoc requests. My boss might say, “Hey, give me a list of every email alias that points to any server except for these 3 exchange servers, then tell me how many emails per day each one of these other mail servers get, and can you get that to me before my next meeting with the executives in, say, 30 minutes?” (grin) Yeah, that actually happens – welcome to IT.
A less geeky example might be image processing. Not all that long ago, instead of using something like gimp when we wanted to do the same set of modifications/filters to some large number of images (or that we frequently had to do), there was a whole library devoted to modifying image files. There were dozens of different programs that would sharpen, blur, convert from one format to another, change color palettes, you name it, they did it. And they all used pipes. So one could make one really long command (or more often put them in a shell script so we didn't have to repeatedly type in log chains of commands) that would call each of these different programs in sequence. One would read from any image format and write a standardized image format all the other commands could read. Then the next might do a sharpen filter. The next might resize it. The next might then export it to some other image format. And if you wrote your shell script correctly you could pipe input into it and pipe it's output to yet another command or script.
See, so honest, it really isn't meant to be arcane. It's meant to be flexible. These gi-normous collections of command line tools aren't arcane to be intimidating, they have a bazillion options because they're trying to be as flexible as possible so you can do more with them.
And most linux users these days will probably never even use the command line tools anymore, which is a little sad. Most will pick a window manager they like and use whatever GUI tools they like the best. And from an end-user's standpoint that's as it should be. But under the hood some of these GUI tools may just be calling other command line programs. And of course power users may spend a little time learning how to use the command line tools and write simple shell scripts to make their lives easier, to automate things they do repeatedly.
So how does one get started? One command that's helpful is the man command (man is short for manual). Want to learn about how the sort command works? Just type “man sort” in a terminal window. Want to see how the uniq command works? “man uniq”. Heck there are even man pages for the man command. I also highly recommend the “nutshell books” from O'Reilly. They have books on anything and everything you can imagine, perl, bash, python (scripting stuff), sendmail, bind, postfix (email and DNS servers), relatively simple stuff to pretty advanced IT topics. Google is also your friend. When I started learning UNIX in the 80s, I quickly had a huge bookshelf full of geeky books on various software and tools. These days most of that info is online if you look around. Not all online resources are as good as the O'Reilly books but hey, it's usually free.
I had someone pass an eye over this and they just asked “Why are unix/linux commands named so… so… weird?” grep? sed? awk? Yeah, as I said they're often written by geeks to be used by geeks. Sometimes the name is related to some technical mechanism they use (grep is short for something like “generic regular expression” if memory serves). Some command/project names are humorous. In ye olden days there was a program called biff which would tell you when you had new email messages. Some said it was short for “be informed” but others said it was named after a dog at a university named Biff Studsworth the 3rd who always barked at the postman. (grin) I don't know which is true but it always made me chuckle. So, yeah, you'll run across some odd project/program names. Just think of it as the price of freedom, of open systems where anyone can contribute and the most useful tools become “standard”.
Addendum… I just remembered that some use the term “linux” to refer only to the kernel and that the operating system is actually a combination of a kernel and the gnu packages. To me, “linux” means the linux kernel and all of the gnu tools (and others) to make up an OS. “OpenBSD” and “FreeBSD” also use the gnu software and a different kernel. (shrug) If you're curious, go read http://gnu.org/ - I mention this just because I appreciate the work of the free software foundation and everyone who's contributed to gnu.