Tag Archives: CPAN

Data Serialization in Perl examples

Leave a reply

One can store data and objects into a file using Serialization. The bytes that is generated can then be re-used to generate an object that will be identical (a clone) to the stored object. We can achieve this by using Storable module from CPAN. The example below will show a way to store and retrieve such objects.

#!/usr/bin/perl

use 5.006;
use strict;
use warnings;
use Data::Dumper;
use Storable;

my %hash1 = ('Test1' => '10',
	     'Test2' => '20');

store %hash1, "serialized_file";
print "##### HASH1 #####n";
print Dumper(%hash1);

my %hash2 = %{retrieve "serialized_file"};
print "n##### HASH2 #####n";
print Dumper(%hash2);

exit 0;

Output:

##### HASH1 #####
$VAR1 = {
          'Test2' => '20',
          'Test1' => '10'
        };

##### HASH2 #####
$VAR1 = {
          'Test2' => '20',
          'Test1' => '10'
        };


For more info see perldoc Storable
Serialization is used to store data and objects into a file for instance. The serie of bytes that is generated can then be re-used to generate an object that will be identical (a clone) to the stored object.

PDL A way to deal with larger arrays

1 Reply

If you are working in scientific domain or using bioperl where you require to deal with bulk data processing then you should seriously consider learning PDL (“Perl Data Language”). PDL is actually a way to deal with larger arrays in Perl.  It allows large N-dimensional data sets such as large images, spectrogram, etc to be stored efficiently and manipulated quickly.

To say it with the words of Karl Glazebrook, initiator of the PDL project:

    “The PDL concept is to give standard perl5 the ability
    to COMPACTLY store and SPEEDILY manipulate the large
    N-dimensional data sets which are the bread and butter
    of scientific computing. e.g. $a=$b+$c can add two
    2048x2048 images in only a fraction of a second.”

 

PDL is well suited for matrix computations, general handling of multidimensional data, image processing, general scientific computation, and numerical applications. It supports I/O for many popular image and data formats including 1D (line plots), 2D (images) and 3D (volume visualization, surface plots via OpenGL) etc.

Latest stable release of PDL is PDL-2.4.7 and can be found at CPAN or Sourceforge.

Happy learning.

Data Serialization in Perl

4 Replies

Before I dig more into details let’s try to understand some basic facts about Serialization :

What exactly Serialization is?

Serialization is the process of converting a data structure or object into a format that can be stored (for example, in a file or memory buffer, or transmitted across a network connection link) and “resurrected” later in the same or another computer environment. When the resulting series of bits is reread according to the serialization format, it can be used to create a semantically identical clone of the original object.

Serialization in Perl:

In Perl, the data which is represented as key-value pairs can be stored in a file. This gives persistence to such data so that the data can be retrieved and manipulated by many independent, but possibly related, programs. However, if the structure of the data is more complex,  associative arrays or hashes do not provide adequate representation. For example, if the data item is an anonymous array of arrays, or a hash of hashes, or an object belonging to a certain class, it is not enough to be able to tie a file to a hash in order to be able to save the data to a file. So, we need to perform serialization in such cases which involves converting the contents of any complex data structure into a string following a certain well-designed algorithm.

The string produced by the serialization algorithm can then be deserialized later in the program when and if necessary. The serialized data structure can be sent over socket connections or used in remote procedure calls as long as the receiving end deserializes it using an appropriate algorithm. The serialized string can be stored to a text file, or a DBM file if it can be somehow associated with a key, or can be stored in a database. The only requirement that before using again, the string should be deserialized. There are several Perl modules that provide for serialization and de-serialization facilities. These include FreezeThaw.pm, Storable.pm, Data::Dumper.pm, Data::Denter.pm XML::Dumper, JSON::XS
etc. A module called Data::Serializer.pm provides a common interface to some of the serialization modules. The default module used by Data::Serializer.pm is Data::Dumper.pm.

In my next post I will try to show some examples.

Installing Perl modules with cpanminus

1 Reply

While browsing CPAN I found an intersting module by Tatsuhiko Miyagawa cpanminus.  Basically, this is a script to get, unpack, build and install modules from CPAN. The best part of  is it’s dependency free, requires zero configuration, and stands alone. When running, it requires only 10MB of RAM. [Source: CPAN]

There are Debian packages, RPMs, FreeBSD ports, and packages for other operation systems available. If you want to use the package management system, search for cpanminus and use the appropriate command to install.

You can also build from latest source itself.

I have tried it on my CentOs 4.6 and Windows XP machine. Some of the advantages which I can see at first place are:

  • It seems to consume lesser memory than traditional like CPAN and CPANPLUS which sometime goes out of the memory for heavy installation.
  • It’s provides really quiet installation in comparison to CPAN. Not many questions.
  • I think it’s good for beginners
  • Automates installation and install dependencies without CPAN installed so no need to download TAR unpack, makefile, make, make test stuff.

Will try to get some  more findings.

Cheers,