- cd PaCE-pipeline/PaCE-clustering
- Preprocessing:
This preprocessing step formats the input FASTA file and generates
the corresponding input datafile for PaCE.
Run the preprocessPaCE command to preprocess the EST input file (fasta format)
eg.,
$ PreprocessPaCE.pl {FASTA data file} {1: prune PolyA/T, 0 otherwise} > tEST.data.PaCE
If the second argument is :
0: Does not modify the input sequences. Just concatenate each sequence
such that each sequence appears in one line and also
converts all DNA characters to upper case.
1: In addition to the two effects produced by option '0' this option
trims off streaks of As and Ts occurring at either ends
of the sequence. This option is not required if the input FASTA
GNU nano 1.2.5 File: PaCE-README
sequences have already been stripped of poly As and Ts or if they
they are required to be input. If you use this option,
make sure you inspect manually some modifications to confirm
there will be no negative impacts on the clustering. This inspection
can be done by first running preprocessPaCE using flag 0 and then
using flag 1 (both on the original file) and taking a diff of both
the output files.
eg.,
$ PreprocessPaCE.pl ../datafiles/tEST.data 0 > ../datafiles/tEST.data.PaCE
This generates another file by name "tEST.data.PaCE" in ../datafiles/ .
- Find "n":
Find the number of sequences in the preprocessed output data file.
Let us call it "n".
This can be found by a simple unix command like:
eg., grep -c ">" ../datafiles/tEST.data.PaCE
ie., n=500 for ../datafiles/tEST.data.PaCE.
- Parameterization:
Parameters to PaCE are kept in the file PaCE.cfg.
Check if PaCE.cfg is there in the directory of the executable.
You typically do NOT need to change any parameters except "window".
If the data size <=10,000 ESTs then a window size of 7 is recommended. If the data size <=30,000 and >=10,000 ESTs then a window size of 8 is recommended.
Otherwise a window size of 9 is recommended.
- Run PaCE:
PaCE takes two parameters:
Usage: {MPIRunCommand} PaCE {preprocessed FASTA datafile} {number of ESTs}
(P.S: Number of processors should be AT LEAST 2)
eg.,
$ mpirun -np 4 PaCE ../datafiles/tEST.data.PaCE 500
where 4 is the number of processors to run on.
"{MPIRunCommand}" depends on the available MPI implementation and job scheduler
of the parallel cluster being used.
For batch mode parallel platforms, use the specified batch submission routines like "qsub" or "llsubmit".
- PaCE output:
The results are of two categories: Run-time and Cluster results.
A summary of these results are printed on the standard output. If the parallel platform
uses batch processing which outputs the standard output to a file, then the summary
will be present in that file.
(i) Run-time results:
The total run-time and the run-time in different components of the system
is shown for each SLAVE processor. This indicates the total run-time for PaCE clustering.
For eg.,
Time taken by slave 1 : Load<0> + Preprocessing Phase<3> + Clustering Phase<1>= 5 secs
Here processor rank 1 took a total of 5 seconds to complete with the run-times in phases
indicated separately. The numbers are truncated to integer values.
Almost all the slave processors take the same amount of total
run-time. Also the time to load the data file into memory
indicated by Load<> first might vary with systems and as it is the
time for initialization, subtracting it from the total run-time
tells the actual time taken by the software.
(ii) Clustering results:
The standard output will also have something like this:
Master: #Clusters Output:= 357 #Singletons=258
Master: #Contained ESTs:= 63
This means: The total number of clusters generated is 357, out of which 258 are singletons.
Also out of the n(=500) ESTs supplied, 63 ESTs are completely contained
(with 100% identity) in other ESTs.
The clusters themselves are located in file estClust.n.p.PaCE
where n is the number of ESTs and p is the number of slave processors.
eg., estClust.500.3.PaCE
PS: The number of slave processors is always one less than the total
number of processors used to run PaCE.
The size distribution of these clusters (in the number of ESTs) is present
in estClustSize.n.p.PaCE.
eg., estClustSize.500.3.PaCE
Each line is of the format "{Cluster#} {Number of Members in Cluster#}".
The set of EST sequences which are contained in (an)other EST sequence(s) are
indicated in ContainedESTs.n.PaCE.
eg., ContainedESTs.500.PaCE
Each line is of the format "{Contained EST sequence header} IN {Container EST sequence header}".
PS: One EST can be contained in multiple EST sequences, and only one container
sequence is indicated here.
Also the set of contained ESTs reported need not comprise of all the ESTs
that are actually contained. More specifically, the contained ESTs
reported are only based on the set for which alignments were performed by PaCE.
The set of EST sequences which are not contained in any other EST sequence in the
provided data set is present in NonContainedESTs.n.PaCE.
eg., NonContainedESTs.500.PaCE
Adding clone mates information :
--------------------------------
As additional input, PaCE is designed to take as input clone mates information.
This information should be present in a file which should be present in
the following format:
>clone mate id #
FASTA header for sequences belonging to this clone mate (each in separate lines)
....
eg., Let the clone mates be in a file myCloneMates that looks like this:
>CloneId1
gi|19863109|
gi|19800130|
>CloneId2
gi|19863111|
gi|19800132|
...
Step 1:
Run the following command (for the above example) :
$perl formatCloneMates.pl myCloneMates tEST.data.PaCE > myCloneMates.PaCE
This will generate the PaCE formatted myCloneMates.PaCE.
(The numbers in each line indicates the sequence number that PaCE
provides for each input sequence.)
Step 2:
The PaCE formatted clone mate file should be in the folder where the
PaCE executable resides.
To specify the clone mate file as input, it has to be provided in the PaCE.cfg file as:
ClonePairsFile clonematefilename
eg.,
ClonePairsFile myCloneMates.PaCE
This has to be done before starting to execute PaCE. PaCE will put each
set of sequences linked by the same clone mate id together in one cluster
in its final output.a
If you do not have any clone pairs information to provide then set:
ClonePairsFile None
PS: Both these steps should be performed for each input FASTA file (even
if you intend to run for subsets of the original FASTA data file).
- Preprocessing:
This preprocessing step formats the input FASTA file and generates
the corresponding input datafile for PaCE.
Run the preprocessPaCE command to preprocess the EST input file (fasta format)
eg.,
$ PreprocessPaCE.pl {FASTA data file} {1: prune PolyA/T, 0 otherwise} > tEST.data.PaCE
If the second argument is :
0: Does not modify the input sequences. Just concatenate each sequence
such that each sequence appears in one line and also
converts all DNA characters to upper case.
1: In addition to the two effects produced by option '0' this option
trims off streaks of As and Ts occurring at either ends
of the sequence. This option is not required if the input FASTA
GNU nano 1.2.5 File: PaCE-README
sequences have already been stripped of poly As and Ts or if they
they are required to be input. If you use this option,
make sure you inspect manually some modifications to confirm
there will be no negative impacts on the clustering. This inspection
can be done by first running preprocessPaCE using flag 0 and then
using flag 1 (both on the original file) and taking a diff of both
the output files.
eg.,
$ PreprocessPaCE.pl ../datafiles/tEST.data 0 > ../datafiles/tEST.data.PaCE
This generates another file by name "tEST.data.PaCE" in ../datafiles/ .
- Find "n":
Find the number of sequences in the preprocessed output data file.
Let us call it "n".
This can be found by a simple unix command like:
eg., grep -c ">" ../datafiles/tEST.data.PaCE
ie., n=500 for ../datafiles/tEST.data.PaCE.
- Parameterization:
Parameters to PaCE are kept in the file PaCE.cfg.
Check if PaCE.cfg is there in the directory of the executable.
You typically do NOT need to change any parameters except "window".
If the data size <=10,000 ESTs then a window size of 7 is recommended. If the data size <=30,000 and >=10,000 ESTs then a window size of 8 is recommended.
Otherwise a window size of 9 is recommended.
- Run PaCE:
PaCE takes two parameters:
Usage: {MPIRunCommand} PaCE {preprocessed FASTA datafile} {number of ESTs}
(P.S: Number of processors should be AT LEAST 2)
eg.,
$ mpirun -np 4 PaCE ../datafiles/tEST.data.PaCE 500
where 4 is the number of processors to run on.
"{MPIRunCommand}" depends on the available MPI implementation and job scheduler
of the parallel cluster being used.
For batch mode parallel platforms, use the specified batch submission routines like "qsub" or "llsubmit".
- PaCE output:
The results are of two categories: Run-time and Cluster results.
A summary of these results are printed on the standard output. If the parallel platform
uses batch processing which outputs the standard output to a file, then the summary
will be present in that file.
(i) Run-time results:
The total run-time and the run-time in different components of the system
is shown for each SLAVE processor. This indicates the total run-time for PaCE clustering.
For eg.,
Time taken by slave 1 : Load<0> + Preprocessing Phase<3> + Clustering Phase<1>= 5 secs
Here processor rank 1 took a total of 5 seconds to complete with the run-times in phases
indicated separately. The numbers are truncated to integer values.
Almost all the slave processors take the same amount of total
run-time. Also the time to load the data file into memory
indicated by Load<> first might vary with systems and as it is the
time for initialization, subtracting it from the total run-time
tells the actual time taken by the software.
(ii) Clustering results:
The standard output will also have something like this:
Master: #Clusters Output:= 357 #Singletons=258
Master: #Contained ESTs:= 63
This means: The total number of clusters generated is 357, out of which 258 are singletons.
Also out of the n(=500) ESTs supplied, 63 ESTs are completely contained
(with 100% identity) in other ESTs.
The clusters themselves are located in file estClust.n.p.PaCE
where n is the number of ESTs and p is the number of slave processors.
eg., estClust.500.3.PaCE
PS: The number of slave processors is always one less than the total
number of processors used to run PaCE.
The size distribution of these clusters (in the number of ESTs) is present
in estClustSize.n.p.PaCE.
eg., estClustSize.500.3.PaCE
Each line is of the format "{Cluster#} {Number of Members in Cluster#}".
The set of EST sequences which are contained in (an)other EST sequence(s) are
indicated in ContainedESTs.n.PaCE.
eg., ContainedESTs.500.PaCE
Each line is of the format "{Contained EST sequence header} IN {Container EST sequence header}".
PS: One EST can be contained in multiple EST sequences, and only one container
sequence is indicated here.
Also the set of contained ESTs reported need not comprise of all the ESTs
that are actually contained. More specifically, the contained ESTs
reported are only based on the set for which alignments were performed by PaCE.
The set of EST sequences which are not contained in any other EST sequence in the
provided data set is present in NonContainedESTs.n.PaCE.
eg., NonContainedESTs.500.PaCE
Adding clone mates information :
--------------------------------
As additional input, PaCE is designed to take as input clone mates information.
This information should be present in a file which should be present in
the following format:
>clone mate id #
FASTA header for sequences belonging to this clone mate (each in separate lines)
....
eg., Let the clone mates be in a file myCloneMates that looks like this:
>CloneId1
gi|19863109|
gi|19800130|
>CloneId2
gi|19863111|
gi|19800132|
...
Step 1:
Run the following command (for the above example) :
$perl formatCloneMates.pl myCloneMates tEST.data.PaCE > myCloneMates.PaCE
This will generate the PaCE formatted myCloneMates.PaCE.
(The numbers in each line indicates the sequence number that PaCE
provides for each input sequence.)
Step 2:
The PaCE formatted clone mate file should be in the folder where the
PaCE executable resides.
To specify the clone mate file as input, it has to be provided in the PaCE.cfg file as:
ClonePairsFile clonematefilename
eg.,
ClonePairsFile myCloneMates.PaCE
This has to be done before starting to execute PaCE. PaCE will put each
set of sequences linked by the same clone mate id together in one cluster
in its final output.a
If you do not have any clone pairs information to provide then set:
ClonePairsFile None
PS: Both these steps should be performed for each input FASTA file (even
if you intend to run for subsets of the original FASTA data file).
No comments:
Post a Comment