February 6, 2023

Blog @ Munaf Sheikh

Latest news from tech-feeds around the world.

6 Advanced Linux command Examples for Beginners to Learn

Hello guys, Linux is an important skill for any developer or DevOps engineer becuase most of the server-side applications run on Linux boxes. At least you should be familiar with essential Linux commands like ls, cat, ps, top, find, grep, awk, sort, uniq, kill, xargs, curl, lsof, etc. I have shared a lot of useful Linux tutorials in this blog and you can take a look at them to learn about those commands in detail. Today, I am going to share some of the useful Linux one-liners you can use in your day-to-day life. These one-liners are either example of one single command or multiple commands used together to do a certain task like finding duplicate rows in a text file or counting how many times each IP address appear in a text file, much like group by clause of SQL.

These commands will help you during troubleshooting, debugging, and data analysis. For example, I have extensively used
grep, cut, sort, and uniq combinations to find out which clients are connecting to our server-side Java application and then pro-actively inform them to move off before we decommission our server and move to the cloud.

The client really appreciated our reports which proactively identified connections from their hosts, which even didn’t know. That’s the power of Linux commands, they are really a must-know tool not just for Java developers but for any programmers.

While there is no end to creativity and how you use a combination of Linux command to solve a problem or extract a piece of information you know, I will share 6 frequently used Linux one-liners which range from finding duplicate rows in a text file to removing ctrl+ M characters from a file which often creep in when you copy a file from Windows to UNIX.

Basic knowledge of Linux commands is necessary to understand these one-liners but if you are a complete beginner then you should first go through a Linux fundamental course like Linux Command line basics, which will teach you each of these commands used here together to do a meaningful task. It’s a very comprehensive yet affordable course and you can buy in just $10 on Udemy flash sales which happen every now and then.

Linux Command One-Liners Examples for Programmers

Without wasting any more of your time, here is my list of some of the useful Linux command one-liners for programmers, software engineers, cloud engineers, and system admins working in the Linux environment.

1. How to Find Duplicate Rows in a Text File 

If you have a text file, you can find lines that are duplicated by running

$ sort file | uniq -d

By default, the uniq command takes a sorted list and prints each line once. If you add the -d option, it only prints lines that occur more than once.

For example,


If you run uniq -d file1, you get:

$ uniq -d file 1

This is because the only bonjour occurs more than once in a row or consequently.

So, to get all duplicates, you need to first sort the file so that all duplicate will come one after other and then you can run sort file1 | uniq -d, which returns:

$ sort data.txt | uniq -d

This one was a very simple example of how you can combine two Linux commands to do some meaningful tasks. If you want to learn more about the sort and uniq command then I suggest you join  The Linux Mastery: Master the Linux Command Line in the 11.5 Hour course on Udemy. This is a project-based course where you will do many such tasks to understand not just these two commands but others like find, grep, awk, and sed.

2. Linux command to remove Ctrl M characters from a file

Strip the CTRL-M (^M), ASCII octal 015, from line-endings from a file. This is useful when DOS (Microsoft Windows etc) originated files are moved to UNIX.

$ tr -d '15' < filename > outputfilename

If you want to learn more about Ctrl-M characters and how they creep into your file while copying from Windows and Linux then check out my earlier post about how to remove Ctrl+M characters from a file in Linux.

3. How to search the jar files in a CLASSPATH for a given Java package name (or class name):

This is a very useful one-liner for Java developers working in the Linux machine. You can use this command to troubleshoot classpath related issues and solving those ClassNotFoundException and NoClassDefFoundError:

$ printenv CLASSPATH | tr : 'n' | grep -i .jar 
                     | xargs -tI {} jar -tf {} | grep org.springframework

To search for something in the path, like all occurences of “dev”:

$ printenv PATH | awk 'BEGIN {RS=":"} /dev/ {print}'

Two ways to look at the PATH (or CLASSPATH etc) with one item on each line:

$ echo $PATH | tr   | grep dev
$ printenv PATH | awk 'BEGIN {RS=":"} {print}'

4. Selective removal of files

I wanted to remove a large number of files from a directory. However, I did not want to descend into subdirectories, nor did I want to remove any .pdf or .chm files. Some of the files had non-standard characters (such as quotes and spaces) in them.

The solution? A simple one-liner using find and xargs

$ find . ( ! -name ".pdf" -a ! -name ".chm" ) 
       -maxdepth 1 -type f -print0 | xargs -0 rm

If you have trouble understanding these commands becuase of several find command options, then I also suggest you check out Linux Command Line Interface (CLI) Fundamentals course on Pluralsight,  a great course to learn essential Linux commands for programmers and developers.

Source link