I have a new application that I need to extract one field, **taskname**, from the main task table (5+ million records) and merge into the subtask table (less 1 million records). When I ran with the 'transaction' statement, I got incomplete results.
Here are my file structures. One main task will have multiple subtasks and each subtask might run in serialized or parallel.
Main taskA (mID, jID, mCPU, taskname, main task beginning/ending time)
subtask1: mID, jID, sCPU, subtask beginning/ending time
subtask2: mID, jID, sCPU, subtask beginning/ending time
. . . . .
I need to search each detailed subtask attributes by task name across different time frame. And, another search will be summarized by time distribution. I know the later one can be achieved with `| timechart sum(sCPU) by taskname span=1h`. But, I got the performance issue for the first search. I did search to use summary indexing and post processing. It seems they suggest to aggregate fields mID and jID. However, I will lose the subtask detailed information. Any suggestion?
↧