NoSQL.RecentChanges
Edit Page
Page Revisions
|
|
SearchWiki NoSQL.RecentChanges Edit Page Page Revisions |
| The easiest and probably most efficient way to convert data in files that were received from an external source is to use the AWK programming language. If the data file to be imported has a fixed field structure then GAWK (GNU-AWK) may be preferable over standard AWK, as the latter can cope easily with fixed-field records. Note that the table header must be inferred manually from the structure of the input file.
The best way to accomplish data conversion is first to construct an xref file from known or observed information in the external data file, then generate the header using 'maketable.' Next, run the AWK script, appending the output to the header file. Finally, run 'istable' to make sure everything went all right.
Another possibility is to use the PERL programming language. A typical PERL script for such a conversion from a file of fixed column data is shown in Table 7. The last two lines do most of the work. Note that this process converts the data into a table body only; the table header still must be generated manually.
Sample Perl script for data conversion:
#!/usr/local/bin/perl
$templ = "A6 A12 x A5 x6 A18 A17 x A38 x4 A2 x A x2 A15 x3 A6 A4 x2" .
" A2 x4 A5 x A3 A3 A4 x26 A12 A12 A12 A29 x7 A3 A3 x6 A6 A6 A8" .
" A4 A3 x3 A3 x3 A8 x10 A2 x4 A12 x138 A6 x66 A24 A36 A12 x6" .
" A6 A6 x6 A6 A24 A12 x50 A42 A42 A42 x34 A12 A12 x186 A6" ;
$0 =~ s-.*/-- ;
$HelpInfo = <<EOH ;
Strip out and reform an 'external' data file into a table.
Usage: $0 [options] file
Options:
-help Print this help information.
Strips out the first 46 fields from an 'external' data file and
reformats it into 'table' format (TAB delimited with NEWLINE at end).
Output is on STDOUT.
EOH
while ( $ARGV[0] =~ /^-/ ) { # Get args
$_ = shift ;
if( /-h.*/ ){ die $HelpInfo ; }
die "Bad arg: $_\n", $HelpInfo ;
}
while(<>){
@a = unpack( $templ, $_ );
print join( "\t", @a), "\n" ;
}
<< ConcatenatingTables | NoSQL.DocumentationIndex | BigTables >> |