perl
The Perl 5 language interpreter
Synopsis
perl
[ -sTtuUWX ]
[ -hv ] [ -V[:configvar] ]
[ -cw ] [ -d[t][:debugger] ] [ -D[number/list] ]
[ -pna ] [ -Fpattern ] [ -l[octal] ] [ -0[octal/hexadecimal] ]
[ -Idir ] [ -m[-]module ] [ -M[-]’module...’ ] [ -f ]
[ -C [number/list] ]
[ -S ]
[ -x[dir] ]
[ -i[extension] ]
[ [-e|-E] ’command’ ] [ -- ] [ programfile ] [ argument ]...
add an example, a script, a trick and tips
examples
source
How do I remove a file in Linux whose name looks like it's ONLY a hyphen, as in "-"
Should be able to do rm -f '-' worked for me in
cygwin
For that matter, rm - also seems to work on red hat.
source
Remove duplicates in each line of a file
Since ruby comes with any Linux distribution I know
of:
ruby -e 'STDIN.readlines.each { |l| l.split(" ").uniq.each { |e| print "#{e} " }; print "\n" }' < test
Here, test is the file that contains the elements.
To explain what this command does—although Ruby can almost be
read from left to right:
- Read the input (which comes from
< test
through your shell)
- Go through each line of the input
- Split the line based on one space separating the items, into
an array (
split(" "))
- Get the unique elements from this array (in-order)
- For each unique element, print it, including a space
(
print "#{e} ")
- Print a newline once we're done with the unique elements
source
Shrinking file size under Linux
You may want to use the
truncate command:
truncate --size=1G test.txt
SIZE can be specified as bytes, KB, K, MB, M, etc. I assume you
can calculate the desired size by hand; if not, you could
probably use the stat command to get information about the file's
current size.
source
unix - split a huge .gz file by line
I'd consider using split.
split a file into pieces
source
When to use Bash and when to use Perl/Python/Ruby?
Bash is a Unix shell it includes a scripting language. It is
rather command processor. you control the way how you run
commands, you actually run them.
Perl/Ruby/Python are general purpose languages.
When you want a shell script, you use Bash
If you want more complex task or not related to shell. Use Python
etc.
I would never compare these languages actually. Python etc. are
portable. You can run them anywhere. Bash is for Unix only.
Python etc. have tons of reusable libraries solving millions of
tasks.
It's almost the same if you ask. "When to use Paint and when to
use Photoshop"
For processing emails I would use Ruby, again, because it has a
lot of reusable libraries.
But the best way would be combine bash and ruby.
That would be right. Like you create a email lrocessor in ruby
and bash script would invoke that ruby script and run other
commans ds.
So whenever you need command processor you use bash. You run unix
commands and control them.
source
Output numbers from a file in a different order
Try doing this in perl
perl -lane '$c=0; for (@F){ print "$F[$c]\t$F[$c+=1]" if $F[$c+1]}' file.txt
Or decomposed :
perl -lane '
$c=0;
for (@F) {
print "$F[$c]\t$F[$c+=1]"
if $F[$c+1];
}
' file.txt
EXPLANATIONS
-
lane switchs means : l=newlines ; a=autosplit in
@F array ; n=like while (<>) magic diamond
operator ; e=basic switch to run a command
-
$c=0 assign 0 to a counter
-
for (@F) { for each element of the current line
-
print "$F[$c]\t$F[$c+=1]" : print array element
with indice $c + tab + $c+1
-
if $F[$c+1]; : apply last line only if $F[$c+1] is
not null
Or using bash (same
algorithm), maybe more human readable for beginners :
while read a; do
arr=( $a )
for ((i=0; i< ${#arr[@]}; i++)); do
[[ ${arr[i+1]} ]] && echo "${arr[i]} ${arr[i+1]}"
done
done < file.txt
source
How do you manage perl modules on a Debian system?
Depending on the modules, I'm fairly sure you can use the apt-get
command to install some of the more common modules.
apt-get install perl5-crypt (maybe its p5-crypt - its been a
while).
It depends a lot on what modules are required though, many of
them are not in the apt packages.
source
convert .dbx (outlook express) mail files to a mbox format (for thunderbird) w/o Windows?
Since no one posted a solution that was implemented nor throughly
documented, I will post my solution.
As above, I simply went to a friends house who has windows
installed and did the conversion the way that everyone else does
it... on windows...
Another way to do this would be to run a virtual machine to do
what you need but I did not have my discs as this wasn't a
solution for me.
source
Shell script to find all types of files in a directory with their count
find . -type f -name '*.*' -exec sh -c 'echo ${0##*.}' {} \; | sort | uniq -c | sort -nr
The echo ${0##*.} gives you the extension of the
file. We pipe the output to sort and then count the
unique lines with uniq.
Some additions:
- Since the output of
uniq is not sorted according
to the number of occurrences, you'll have to pipe again into a
numeric sort (-nr) if you want it sorted.
- If you want to search your current directory only, add
-maxdepth 1 to your find command.
- Pipe into
awk '{print $2, $1}' to show the count
after the extensions.
source
Renaming files in batch with Unix RENAME
rename doesn't do sed-style
substitutions. This very short Perl script will let you do
regmv *.sam s/indel/snp/:
#!/usr/bin/perl -w
#
# regmv - Rename files using a regular expression
#
# This program renames files by using a regular expression to
# determine their new name.
#
# Usage: regmv file [file ...] regexp
#
# $Id: regmv,v 1.1 1998/10/14 17:07:55 blrfl Exp $
#
sub at_clean()
{
my $message = $@;
$message =~ s|\s+at /.*$||s;
return $message;
}
(@ARGV > 1)
|| die "Usage: regmv [options] file [file ...] regexp\n";
my $expr = pop @ARGV;
$_ = 'SomeValue';
eval "\$_ =~ $expr";
die "Yuck: " . at_clean() if $@;
foreach (@ARGV)
{
my $old = $_;
eval "\$_ =~ $expr";
next if $_ eq $old;
(rename $old, $_)
|| die "Rename failed: $!\n";
}
source
Linux / Perl - What happens when a process is forked?
From Advanced Programming in the UNIX
Environment by W. Richard Stevens (pg. 188):
8.3 fork function
The only way a new process is created by the Unix
kernel is when an existing process calls the fork
function. (This doesn't apply to the special processes that we
mentioned in the previous section—the swapper,
init, and pagedaemon. These processes are created
specially by the kernel as part of bootstrapping.)
#include <sys/types.h>
#include <unistd.h>
pid_t fork(void);
/* Returns: 0 in child, process ID of child in parent, -1 on error */
The new process created by fork is called the
child process. The function is called once but returns
twice. The only difference in the returns is that the return
value in the child is 0 while the return value in the parent is
the process ID of the new child. The reason the child's process
ID is returned to the parent is because a process can have more
than one child, so there is no function that allows a process
to obtain the process IDs of its children. The reason
fork returns 0 to the child is because a process
can have only a single parent, so the child can always call
getppid to obtain the process ID of its parent.
(Process ID 0 is always in use by the swapper, so it's not
possible for 0 to be the process ID of a child.)
Both the child and parent continue executing with the
instruction that follows the call to fork. The
child is a copy of the parent. For example, the child gets a
copy of the parent's data space, heap, and stack. Note that
this is a copy for the child—the parent and child do not share
these portions of memory. Often the parent and child share the
text segment (Section 7.6), if it is read-only.
On Linux, Perl's fork operator calls the system's
fork and returns undef
on failure rather than -1.
Stevens gives lists (pg. 192) of inherited properties and
differences between parents processes and their forked children:
Besides open files, there are numerous other properties of the
parent that are inherited by the child:
- real user ID, real group ID, effective user ID, effective
group ID
- supplementary group IDs
- process group ID
- session ID
- controlling terminal
- set-user-ID flag and set-group-ID flag
- current working directory
- root directory
- file mode creation mask
- signal mask and dispositions
- the close-on-exec flag for any open file descriptors
- environment
- attached shared memory segments
- resource limits
The differences between the parent and child are
- the return value from
fork
- the process IDs are different
- the two processes have different parent process IDs—the
parent process ID of the child is the parent; the parent
process ID of the parent doesn't change
- the child's values for
tms_utime,
tms_stime, tms_cutime, and
tms_ustime are set to 0
- file locks set by the parent are not inherited by the child
- pending alarms are cleared for the child
- the set of pending signals for the child is set to the
empty set
source
How can I identify non-ASCII characters from the shell?
$ perl -ne 'print "$. $_" if m/[\x80-\xFF]/' utf8.txt
2 Pour être ou ne pas être
4 By? ?i neby?
5 ???
or
$ grep -n -P '[\x80-\xFF]' utf8.txt
2:Pour être ou ne pas être
4:By? ?i neby?
5:???
where utf8.txt is
$ cat utf8.txt
To be or not to be.
Pour être ou ne pas être
Om of niet zijn
By? ?i neby?
???
source
How can uptime be used to trigger an action?
You can easily determine the uptime in days with awk:
# Print days of uptime, or zero if less than 1 day.
uptime | awk '/days?/ {print $3; next}; {print 0}'
You can use this with command substitution to perform any action
you like based on the results. For example:
#!/bin/bash
days () { uptime | awk '/days?/ {print $3; next}; {print 0}'; }
UPTIME_THRESHOLD=200
if [ $(days) -ge $UPTIME_THRESHOLD ]; then
: # Take some action.
fi
Obviously, the action you take is up to you. You can mail
yourself messages, schedule a reboot with the at
command, or anything else that you feel necessary to do.
You may also wish to set this script up as a daily cron job, so
that it will trigger your defined action when the uptime
threshold has been exceeded. If you have root access, you could
simply drop the script into /etc/cron.daily/, or you
might set up your personal crontab to call this script once a
day.
source
Force wget proxy to non environment proxy
You can set the http_proxy environment variable just
for a specific invocation of wget:
http_proxy=http://specific-squid-proxy-host:3129/ wget http://server/page
description
Perl officially
stands for Practical Extraction and Report Language, except
when it doesn’t.
Perl was
originally a language optimized for scanning arbitrary text
files, extracting information from those text files, and
printing reports based on that information. It quickly
became a good language for many system management tasks.
Over the years, Perl has grown into a general-purpose
programming language. It’s widely used for everything
from quick "one-liners" to full-scale application
development.
The language is
intended to be practical (easy to use, efficient, complete)
rather than beautiful (tiny, elegant, minimal).
Perl combines
(in the author’s opinion, anyway) some of the best
features of C, sed, awk, and sh, so
people familiar with those languages should have little
difficulty with it. (Language historians will also note some
vestiges of csh, Pascal, and even BASIC-PLUS.)
Expression syntax corresponds closely to C expression
syntax. Unlike most Unix utilities, Perl does not
arbitrarily limit the size of your data--if you’ve got
the memory, Perl can slurp in your whole file as a single
string. Recursion is of unlimited depth. And the tables used
by hashes (sometimes called "associative arrays")
grow as necessary to prevent degraded performance. Perl can
use sophisticated pattern matching techniques to scan large
amounts of data quickly. Although optimized for scanning
text, Perl also has many excellent tools for slicing and
dicing binary data.
But wait,
there’s more...
Begun in 1993
(see perlhist), Perl version 5 is nearly a complete rewrite
that provides the following additional benefits:
•
modularity and reusability using innumerable modules
Described in
perlmod, perlmodlib, and perlmodinstall.
•
embeddable and extensible
Described in
perlembed, perlxstut, perlxs, perlcall, perlguts, and
xsubpp.
•
roll-your-own magic variables
(including multiple simultaneous DBM
implementations)
Described in
perltie and AnyDBM_File.
•
subroutines can now be
overridden, autoloaded, and prototyped
Described in
perlsub.
•
arbitrarily nested data
structures and anonymous functions
Described in
perlreftut, perlref, perldsc, and perllol.
•
object-oriented programming
Described in
perlobj, perlboot, perltoot, perltooc, and perlbot.
•
support for light-weight
processes (threads)
Described in
perlthrtut and threads.
•
support for Unicode,
internationalization, and localization
Described in
perluniintro, perllocale and Locale::Maketext.
Described in
perlsub.
•
regular expression
enhancements
Described in
perlre, with additional examples in perlop.
•
enhanced debugger and
interactive Perl environment, with integrated editor
support
Described in
perldebtut, perldebug and perldebguts.
•
POSIX 1003.1
compliant library
Described in
POSIX .
Okay,
that’s definitely enough hype.
availability
Perl is available for most operating systems, including virtually
all Unix-like platforms. See "Supported Platforms" in perlport
for a listing.
diagnostics
The "use warnings" pragma (and the -w switch)
produces some lovely diagnostics.
See perldiag for explanations of all Perl’s diagnostics. The
"use diagnostics" pragma automatically turns Perl’s
normally terse warnings and errors into these longer forms.
Compilation errors will tell you the line number of the error,
with an indication of the next token or token type that was to be
examined. (In a script passed to Perl via -e switches,
each -e is counted as one line.)
Setuid scripts have additional constraints that can produce error
messages such as "Insecure dependency". See perlsec.
Did we mention that you should definitely consider using the
-w switch?
environment
See perlrun.
files
"@INC" locations of perl libraries
getting help
The perldoc program gives you access to all the
documentation that comes with Perl. You can get more
documentation, tutorials and community support online at
<http://www.perl.org/>.
If you’re new to Perl, you should start by running "perldoc
perlintro", which is a general intro for beginners and
provides some background to help you navigate the rest of Perl’s
extensive documentation. Run "perldoc perldoc" to learn
more things you can do with perldoc.
For ease of access, the Perl manual has been split up into
several sections.
Overview
perl Perl overview (this section)
perlintro Perl introduction for beginners
perltoc Perl documentation table of contents
Tutorials
perlreftut Perl references short introduction
perldsc Perl data structures intro
perllol Perl data structures: arrays of arrays
perlrequick Perl regular expressions quick start
perlretut Perl regular expressions tutorial
perlboot Perl OO tutorial for beginners
perltoot Perl OO tutorial, part 1
perltooc Perl OO tutorial, part 2
perlbot Perl OO tricks and examples
perlperf Perl Performance and Optimization Techniques
perlstyle Perl style guide
perlcheat Perl cheat sheet
perltrap Perl traps for the unwary
perldebtut Perl debugging tutorial
perlfaq Perl frequently asked questions
perlfaq1 General Questions About Perl
perlfaq2 Obtaining and Learning about Perl
perlfaq3 Programming Tools
perlfaq4 Data Manipulation
perlfaq5 Files and Formats
perlfaq6 Regexes
perlfaq7 Perl Language Issues
perlfaq8 System Interaction
perlfaq9 Networking
Reference Manual
perlsyn Perl syntax
perldata Perl data structures
perlop Perl operators and precedence
perlsub Perl subroutines
perlfunc Perl built-in functions
perlopentut Perl open() tutorial
perlpacktut Perl pack() and unpack() tutorial
perlpod Perl plain old documentation
perlpodspec Perl plain old documentation format specification
perlpodstyle Perl POD style guide
perlrun Perl execution and options
perldiag Perl diagnostic messages
perllexwarn Perl warnings and their control
perldebug Perl debugging
perlvar Perl predefined variables
perlre Perl regular expressions, the rest of the story
perlrebackslash Perl regular expression backslash sequences
perlrecharclass Perl regular expression character classes
perlreref Perl regular expressions quick reference
perlref Perl references, the rest of the story
perlform Perl formats
perlobj Perl objects
perltie Perl objects hidden behind simple variables
perldbmfilter Perl DBM filters
perlipc Perl interprocess communication
perlfork Perl fork() information
perlnumber Perl number semantics
perlthrtut Perl threads tutorial
perlport Perl portability guide
perllocale Perl locale support
perluniintro Perl Unicode introduction
perlunicode Perl Unicode support
perlunifaq Perl Unicode FAQ
perluniprops Index of Unicode Version 6.0.0 properties in Perl
perlunitut Perl Unicode tutorial
perlebcdic Considerations for running Perl on EBCDIC platforms
perlsec Perl security
perlmod Perl modules: how they work
perlmodlib Perl modules: how to write and use
perlmodstyle Perl modules: how to write modules with style
perlmodinstall Perl modules: how to install from CPAN
perlnewmod Perl modules: preparing a new module for distribution
perlpragma Perl modules: writing a user pragma
perlutil utilities packaged with the Perl distribution
perlcompile Perl compiler suite intro
perlfilter Perl source filters
perlglossary Perl Glossary
Internals and C Language Interface
perlembed Perl ways to embed perl in your C or C++ application
perldebguts Perl debugging guts and tips
perlxstut Perl XS tutorial
perlxs Perl XS application programming interface
perlclib Internal replacements for standard C library functions
perlguts Perl internal functions for those doing extensions
perlcall Perl calling conventions from C
perlmroapi Perl method resolution plugin interface
perlreapi Perl regular expression plugin interface
perlreguts Perl regular expression engine internals
perlapi Perl API listing (autogenerated)
perlintern Perl internal functions (autogenerated)
perliol C API for Perl's implementation of IO in Layers
perlapio Perl internal IO abstraction interface
perlhack Perl hackers guide
perlsource Guide to the Perl source tree
perlinterp Overview of the Perl intepreter source and how it works
perlhacktut Walk through the creation of a simple C code patch
perlhacktips Tips for Perl core C code hacking
perlpolicy Perl development policies
perlgit Using git with the Perl repository
Miscellaneous
perlbook Perl book information
perlcommunity Perl community information
perltodo Perl things to do
perldoc Look up Perl documentation in Pod format
perlhist Perl history records
perldelta Perl changes since previous version
perl5141delta Perl changes in version 5.14.1
perl5140delta Perl changes in version 5.14.0
perl51311delta Perl changes in version 5.13.11
perl51310delta Perl changes in version 5.13.10
perl5139delta Perl changes in version 5.13.9
perl5138delta Perl changes in version 5.13.8
perl5137delta Perl changes in version 5.13.7
perl5136delta Perl changes in version 5.13.6
perl5135delta Perl changes in version 5.13.5
perl5134delta Perl changes in version 5.13.4
perl5133delta Perl changes in version 5.13.3
perl5132delta Perl changes in version 5.13.2
perl5131delta Perl changes in version 5.13.1
perl5130delta Perl changes in version 5.13.0
perl5123delta Perl changes in version 5.12.3
perl5122delta Perl changes in version 5.12.2
perl5121delta Perl changes in version 5.12.1
perl5120delta Perl changes in version 5.12.0
perl5115delta Perl changes in version 5.11.5
perl5114delta Perl changes in version 5.11.4
perl5113delta Perl changes in version 5.11.3
perl5112delta Perl changes in version 5.11.2
perl5111delta Perl changes in version 5.11.1
perl5110delta Perl changes in version 5.11.0
perl5101delta Perl changes in version 5.10.1
perl5100delta Perl changes in version 5.10.0
perl595delta Perl changes in version 5.9.5
perl594delta Perl changes in version 5.9.4
perl593delta Perl changes in version 5.9.3
perl592delta Perl changes in version 5.9.2
perl591delta Perl changes in version 5.9.1
perl590delta Perl changes in version 5.9.0
perl589delta Perl changes in version 5.8.9
perl588delta Perl changes in version 5.8.8
perl587delta Perl changes in version 5.8.7
perl586delta Perl changes in version 5.8.6
perl585delta Perl changes in version 5.8.5
perl584delta Perl changes in version 5.8.4
perl583delta Perl changes in version 5.8.3
perl582delta Perl changes in version 5.8.2
perl581delta Perl changes in version 5.8.1
perl58delta Perl changes in version 5.8.0
perl573delta Perl changes in version 5.7.3
perl572delta Perl changes in version 5.7.2
perl571delta Perl changes in version 5.7.1
perl570delta Perl changes in version 5.7.0
perl561delta Perl changes in version 5.6.1
perl56delta Perl changes in version 5.6
perl5005delta Perl changes in version 5.005
perl5004delta Perl changes in version 5.004
perlartistic Perl Artistic License
perlgpl GNU General Public License
Language-Specific
perlcn Perl for Simplified Chinese (in EUC-CN)
perljp Perl for Japanese (in EUC-JP)
perlko Perl for Korean (in EUC-KR)
perltw Perl for Traditional Chinese (in Big5)
Platform-Specific
perlaix Perl notes for AIX
perlamiga Perl notes for AmigaOS
perlbeos Perl notes for BeOS
perlbs2000 Perl notes for POSIX-BC BS2000
perlce Perl notes for WinCE
perlcygwin Perl notes for Cygwin
perldgux Perl notes for DG/UX
perldos Perl notes for DOS
perlepoc Perl notes for EPOC
perlfreebsd Perl notes for FreeBSD
perlhaiku Perl notes for Haiku
perlhpux Perl notes for HP-UX
perlhurd Perl notes for Hurd
perlirix Perl notes for Irix
perllinux Perl notes for Linux
perlmacos Perl notes for Mac OS (Classic)
perlmacosx Perl notes for Mac OS X
perlmpeix Perl notes for MPE/iX
perlnetware Perl notes for NetWare
perlopenbsd Perl notes for OpenBSD
perlos2 Perl notes for OS/2
perlos390 Perl notes for OS/390
perlos400 Perl notes for OS/400
perlplan9 Perl notes for Plan 9
perlqnx Perl notes for QNX
perlriscos Perl notes for RISC OS
perlsolaris Perl notes for Solaris
perlsymbian Perl notes for Symbian
perltru64 Perl notes for Tru64
perluts Perl notes for UTS
perlvmesa Perl notes for VM/ESA
perlvms Perl notes for VMS
perlvos Perl notes for Stratus VOS
perlwin32 Perl notes for Windows
On Debian systems, you need to install the perl-doc
package which contains the majority of the standard Perl
documentation and the perldoc program.
Extensive additional documentation for Perl modules is available,
both those distributed with Perl and third-party modules which
are packaged or locally installed.
You should be able to view Perl’s documentation with your
man(1) program or perldoc(1).
In general, if something strange has gone wrong with your program
and you’re not sure where you should look for help, try the
-w switch first. It will often point out exactly where the
trouble is.
notes
The Perl motto is "There’s more than one way to do it." Divining
how many more is left as an exercise to the reader.
The three principal virtues of a programmer are Laziness,
Impatience, and Hubris. See the Camel Book for why.
bugs
The
-w switch is not mandatory.
Perl is at the
mercy of your machine’s definitions of various
operations such as type casting, atof(), and
floating-point output with sprintf().
If your stdio
requires a seek or eof between reads and writes on a
particular stream, so does Perl. (This doesn’t apply
to sysread() and syswrite().)
While none of
the built-in data types have any arbitrary size limits
(apart from memory size), there are still a few arbitrary
limits: a given variable name may not be longer than 251
characters. Line numbers displayed by diagnostics are
internally stored as short integers, so they are limited to
a maximum of 65535 (higher numbers usually being affected by
wraparound).
You may mail
your bug reports (be sure to include full configuration
information as output by the myconfig program in the perl
source tree, or by "perl -V") to
perlbug[:at:]perl[:dot:]org . If you’ve succeeded in compiling
perl, the perlbug script in the utils/ subdirectory
can be used to help mail in a bug report.
Perl actually
stands for Pathologically Eclectic Rubbish Lister, but
don’t tell anyone I said that.
see also
http://www.perl.org/ the Perl homepage
http://www.perl.com/ Perl articles (O'Reilly)
http://www.cpan.org/ the Comprehensive Perl Archive
http://www.pm.org/ the Perl Mongers
author
Larry Wall
<larry[:at:]wall[:dot:]org>, with the help of oodles of other
folks.
If your Perl
success stories and testimonials may be of help to others
who wish to advocate the use of Perl in their applications,
or if you wish to simply express your gratitude to Larry and
the Perl developers, please write to
perl-thanks[:at:]perl[:dot:]org .