Quantcast
Channel: UNIX and Linux Forums
Viewing all articles
Browse latest Browse all 16232

Need a Python Script to filter huge files

$
0
0
I work on various messages received from server and want to write a python script that can sort messages with unique flag values and give me the output in a text file.

I get these messages in the form of .zcap file from server and I use an internal tool to filter:

Step 1) Zcap file to get security types,
Step 2) then filter each security types file that generate various exchanges files
Steps 3) lastly filter each exchange files to get message types files.

For eg: I filtered a .zcap file to get message types as –
BOND-----CVE----- MTR_BOND, MTFD_BOND, MTQ_BQUOTE, MTQ_MBBOQUOTE
BOND-----NYSE---- MTR_BOND, MTFD_BOND, MTQ_BQUOTE
BOND-----TSE--- MTR_BOND, MTFD_BOND, MTA_RECAP, MTT_TRADE, MTT_STATUS, MTA_CLOSE

Step 4) at present, I run a UNIX command on each message type that generates multiple text files of each unique flag value messages (for eg; MTR_BOND).
$ awk '/MTQ/,/Quote Condition/{a[i++]=$0;if($0~/Flags:/){sub(":","",$2);fname=$2}if($0~/Quote Condition/){for(j=0;j<=i;j++)print a[j] > fname;i=0}}' MTA.txt

This is a very tedious job to run this command on each message type file (for eg; MTR_BOND) level, I want a python script that I run at Security Type level (for eg; BOND) .
Here I attach text file for BOND and its exchanges and message types. Please write a python script (preferably) to generate 1 text file for each message types with only 1 message of unique flag value.

Attached Files
File Type: txt BOND.txt (541.8 KB)
File Type: txt CVE_exchange.txt (13.3 KB)
File Type: txt NYSE_exchange.txt (489.5 KB)
File Type: txt TSE_exchange.txt (40.0 KB)

Viewing all articles
Browse latest Browse all 16232

Trending Articles