Linux · Systems fluency
How do you process text with awk, sed, and friends?
Short answer
Use awk for field-based records and calculations, sed for stream editing and substitutions, and tools such as cut, sort, uniq, tr, and jq for focused transformations. Compose them in pipelines.
The Unix text toolkit is designed around streams. Each tool does one transformation and passes the result on. awk is a small programming language for records and fields; sed is a line-oriented editor; jq handles JSON specifically.
awk -F: '{print $1}' /etc/passwd | head
sed -n '1,10p' app.log
cut -d, -f2,4 report.csv | sort | uniq -c
jq -r '.items[].name' response.jsonChoosing the tool
- awk: columns, sums, filters, and formatted reports
- sed: substitutions, deletions, and line-range edits
- cut: simple delimiter-based field selection
- sort and uniq: ordering and counting adjacent duplicates
- jq: structured JSON extraction and transformation
sed -i edits files in place, but the backup suffix syntax differs between GNU and BSD sed. On production files, take a backup and test the expression on a copy first. CSV with quoted delimiters is not safe for cut or naive awk field splitting.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1When would you use awk instead of cut?
- 2What are the portability issues with sed -i?
- 3How do you count duplicate lines?
- 4Why is parsing CSV with cut risky?
Say it in the interview
I pick the smallest tool that expresses the transformation: awk for fields and calculations, sed for substitutions, cut for simple columns, and jq for JSON. I compose them in pipelines and test on a copy before in-place edits. For CSV with quoting I use a real parser rather than cut or naive awk.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live Linux mock interview and get scored feedback on depth, clarity and confidence.