- tr
The Unix program tr is used to translate between two sets of characters. Characters specified in one set are converted to the matching character in the second set. Thus, to convert the Ctrl-m of a Mac OS text file to the line feed (Ctrl-j) of a Unix text file, at the Unix command line, enter:
tr '\r' '\n' < macfile.txt > unixfile.txtHere, \r and \n are special escape sequences that tr interprets as Ctrl-m (a carriage return) and Ctrl-j (a line feed), respectively. Thus, to convert a Unix text file to a Mac OS text file, enter:
tr '\n' '\r' < unixfile.txt > macfile.txtNote: The escape sequences must be surrounded by quotes for these commands to work.
To replace all letters, numbers, parenthesis and "_" and "^" by "-" in
the file head enter :
nawk '{ gsub(/[a-zA-Z0-9"^""_""("")"]/, "+");print $0;}' head.csv
To use awk to convert a Mac OS file to Unix, at the Unix prompt, enter:
awk '{ gsub("\r", "\n"); print $0;}' macfile.txt > unixfile.txtTo convert a Unix file to Mac OS using awk, at the command line, enter:
awk '{ gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txtOn some systems, the version of awk may be old and not include the function gsub. If so, try the same command, but replace awk with gawk or nawk.
To replace the remove all lines starting with #, enter :
awk '{if ( substr($0,1,1)!="#") print $0}' list_2mass.tsv
To replace the remove all lines starting with either # or carriage
return or SPACE, enter :
awk '{if (substr($0,1,1)!="#"&&substr($0,1,1)!='\n'&&substr($0,1,1)!=" ") print $0}' list_2mass.tsv
It can also be done with egrep
egrep -v '(^\#|^ $|^$|^ $)' < list2.tsv
14 of the most common sed commands for one-line use.
- Replace a newline to \\followed by a newline
- Double space a file
- Triple space a file
- Under UNIX: convert DOS newlines (CR/LF) to Unix format
sed 's/^M$// file # in bash/tcsh, press Ctrl-V then Ctrl-M
- Under DOS: convert Unix newlines (LF) to DOS format
sed -n p file # method 2
- Delete leading whitespace (spaces/tabs) from front of each line
# character. Under bash or tcsh, press Ctrl-V then Ctrl-I.
sed 's/^[ ^t]*//' file
- Delete trailing whitespace (spaces/tabs) from end of each line
- Delete BOTH leading and trailing whitespace from each line
- Substitute "foo" with "bar" on each line
sed 's/foo/bar/4' file # replaces only 4th instance in a line
sed 's/foo/bar/g' file # replaces ALL instances within a line
- Substitute "foo" with "bar" ONLY for lines which contain "baz"
- Delete all CONSECUTIVE blank lines from file except the first.
# (emulates "cat -s")
sed '/./,/^$/!d' file # this allows 0 blanks at top, 1 at EOF
sed '/^$/N;/\n$/D' file # this allows 1 blank at top, 0 at EOF
- Delete all leading blank lines at top of file (only).
- Delete all trailing blank lines at end of file (only).
- If a line ends with a backslash, join the next line to it.
- If a line begins with an equal sign, append it to the
sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file
To convert a Mac OS text file to a Unix text file using Perl, at the Unix shell prompt, enter:
perl -p -e 's/\r/\n/g' < macfile.txt > unixfile.txtTo convert from a Unix text file to a Mac OS text file with Perl, at the Unix shell prompt, enter:
perl -p -e 's/\n/\r/g' < unixfile.txt > macfile.txt