This is the fourth and final blog post in this series. In this last entry, we will be taking everything we’ve written so far and adding to it a couple of new functions to allow us to take two separate XFR files and merge the instance files together into a single file. In the end, we’ve allowed the user to select two files to compare, altered the currencies as necessary, merged the files, and exported the instance files into a single document.
Friday, May 12. 2017
LDC #34: XBRL Merger, Part 4
Going forward from last week, the changes / additions made to the script are:
1) Three new defines for instance file line types.
2) The function get_line_type has been added. This function reads a line from our instance file and returns one of our three new defines based on the type of the line.
3) The function copy_files has been added. This function copies files from the master folder into our target folder. It skips over the instance file, however, because that will be generated by the write_instance function.
4) The function write_instance has been added. This is the big one. It takes our data structures created by the read_file_contents function, examines them, and writes out a file that contains all data from both structures without duplicates.
5) The run function has been slightly modified. We need to update it to actually run these functions and remove our TODO comments from last week.
#include "XBRLMerger.rc"
#define EXPORT_FOLDER_PREFIX "Merge"
#define CONTEXT_START "<xbrli:context"
#define CONTEXT_END "</xbrli:context>"
#define UNIT_START "<xbrli:unit"
#define UNIT_END "</xbrli:unit>"
#define FACT_IDENTIFIER "contextRef"
#define ERROR_WARN (ERROR_SOFT | 0x00005555)
#define FILE_ONE 1
#define FILE_TWO 2
#define CONTEXT 3
#define UNIT 4
#define FACT 5
#define OTHER 999
#define DEBUG false
#define DEBUG_MSG_MAX 150
string edit_windows[][];
int bigger_file;
int namesize_dif;
string FileOneContexts[][];
string FileTwoContexts[][];
string FileOneUnits[][];
string FileTwoUnits[][];
string FileOneFacts[];
string FileTwoFacts[];
string FileOne, FileTwo;
handle FileOneWindow;
handle FileTwoWindow;
int run (int f_id, string mode);
int validate_file (string file, string display);
string export_file (string foldersuffix, string file, handle window);
string get_export_folder (string file, string foldersuffix);
int compare_xbrl (string instance,string f1folder, string f2folder);
int read_file_contents (int file, string instance);
int clear_folder (string path);
int copy_files (string instance, string source, string dest);
int write_instance (string instance, string master, string dest, int bigger);
int get_line_type (string line);
int debug_message (string message);
int compare_filesizes (string f1, string f2);
int print_tables ();
/****************************************/
int setup() { /* Called from Application Startup */
/****************************************/
string fnScript; /* Us */
string item[10]; /* Menu Item */
int rc; /* Return Code */
/* */
/* ** Add Menu Item */
/* * Common */
item["Class"] = "Extension"; /* Function Code */
/* o Define Function */
item["Code"] = "XBRL_MERGE"; /* Function Code */
item["MenuText"] = "&Merge XFR Files"; /* Menu Text */
item["Description"] = "<B>Merge XBRL Files</B>"; /* description */
item["Description"].= "\r\rMerges two XBRL Instance Files."; /* description */
/* o Check for Existing */
rc = MenuFindFunctionID(item["Code"]); /* Look for existing */
if (IsNotError(rc)) { /* Was already be added */
return ERROR_NONE; /* Exit */
} /* end error */
/* o Registration */
rc = MenuAddFunction(item); /* Add the item */
if (IsError(rc)) { /* Was already be added */
return ERROR_NONE; /* Exit */
} /* end error */
fnScript = GetScriptFilename(); /* Get the script filename */
MenuSetHook(item["Code"], fnScript, "run"); /* Set the Hook */
return ERROR_NONE; /* Return value (does not matter) */
} /* end setup */
/****************************************/
int run(f_id,mode){ /* main run loop */
/****************************************/
string errmsg; /* an error message */
string master_folder; /* folder for master files */
string merge_output; /* the output folder for merge */
string f1folder,f2folder; /* folders that were exported to */
string f1instance,f2instance; /* instance files of exported XFRs */
qword f1namesize, f2namesize; /* sizes of filenames */
int sizeres; /* result of filesize compare */
int rc; /* result */
/* */
bigger_file = OTHER; /* set default value for bigger file */
if (mode!="preprocess"){ /* if not in preprocess */
return ERROR_NONE; /* return no error */
} /* */
edit_windows = EnumerateEditWindows(); /* get open edit windows */
rc = DialogBox("MergeXBRLDlg", "merge_"); /* open selector dialog */
if (IsError(rc)==true){ /* if the user didn't press OK */
CloseHandle(FileOneWindow); /* close handle */
CloseHandle(FileTwoWindow); /* close handle */
return rc; /* return */
} /* */
/* */
ProgressOpen("XBRL Merger"); /* open progress */
ProgressSetStatus("Exporting Files"); /* set status */
ProgressUpdate(1,10); /* update progress */
f1folder = get_export_folder(FileOne,"One"); /* get the file 1 folder */
errmsg = GetLastErrorMessage(); /* get the last error */
if (f1folder == "" ){ /* if we cannot get the folder */
MessageBox('x',"Cannot create folder for export. %s",errmsg); /* display error message */
return ERROR_EXIT; /* return with error */
} /* */
f2folder = get_export_folder(FileTwo,"Two"); /* get the file 2 folder */
errmsg = GetLastErrorMessage(); /* */
if (f2folder == ""){ /* if we cannot get a folder */
MessageBox('x',"Cannot create folder for export. %s",errmsg); /* display error message */
return ERROR_EXIT; /* return with error */
} /* */
f1instance = export_file("One",FileOne, FileOneWindow); /* export the first file */
f2instance = export_file("Two",FileTwo, FileTwoWindow); /* export the second file */
ProgressSetStatus("Comparing XBRL files"); /* progress set status */
ProgressUpdate(2,10); /* update progress status */
if (f1instance=="" || f2instance==""){ /* test if either instance is blank */
MessageBox('x',"Unable to export XFR files. "+errmsg); /* display error message */
return ERROR_EXIT; /* return with error */
} /* */
rc = compare_xbrl(f1instance,f1folder,f2folder); /* check if the files can be merged */
errmsg = GetLastErrorMessage(); /* get the last error message */
if (IsError(rc)){ /* if there was a problem */
if (rc==ERROR_WARN){ /* maybe not fatal, ask user to cont. */
rc = YesNoBox('q',"File size mismatch, result may have errors."+/* ask user */
errmsg+" Continue?"); /* ask user */
if (rc!=IDYES){ /* if the user idn't press yes */
return ERROR_EXIT; /* return with error */
} /* */
} /* */
else{ /* if the error is definitely fatal */
MessageBox('x',"Files are not compatible to merge. %s", errmsg);/* display error */
return rc; /* return eror code */
} /* */
} /* */
master_folder = f1folder; /* set default master folder */
if (bigger_file == FILE_TWO){ /* if file 2 is bigger */
master_folder = f2folder; /* if file 2 is the master folder */
} /* */
if (bigger_file == OTHER){ /* if we don't know which file is bigger*/
sizeres = compare_filesizes(AddPaths(f1folder,f1instance), /* get result of size compare */
AddPaths(f2folder,f2instance)); /* get the result of the size compare */
bigger_file = FILE_ONE; /* set bigger file */
if (sizeres == FILE_TWO){ /* if file two is bigger */
bigger_file = FILE_TWO; /* set file two as bigger file */
master_folder = f2folder; /* set file two as master file */
} /* */
} /* */
debug_message(FormatString("Master File: %d",bigger_file)); /* add debug message */
debug_message(FormatString("Master Folder: %s",master_folder)); /* add debug message */
merge_output = f1folder; /* set output folder */
if (bigger_file == FILE_TWO){ /* if file two is bigger */
merge_output = f2folder; /* set output folder to file two */
} /* */
merge_output = AddPaths(merge_output,"../"+ /* set path for new merged folder */
EXPORT_FOLDER_PREFIX+"Combined"); /* set path for new merged folder */
if (IsFolder(merge_output)==false){ /* if output folder doesn't exist */
CreateFolder(merge_output); /* create our output folder */
} /* */
rc = GetLastError(); /* get the last error */
if (IsError(rc)){ /* if we cannot create our folder */
MessageBox('x',"Folder %s does not exist and cannot be created."+ /* display error */
"Error: %0x", merge_output); /* display error */
return ERROR_EXIT; /* return with error */
} /* */
rc = clear_folder(merge_output); /* make sure it's empty */
if (IsError(rc)){ /* check if clear_folder worked */
MessageBox('x',"Cannot delete files in folder %s. Error: %0x", /* display error */
merge_output,rc); /* display error */
return rc; /* return with erro */
} /* */
ProgressSetStatus("Reading Files"); /* update status */
ProgressUpdate(4,10); /* update progress */
read_file_contents(FILE_ONE,AddPaths(f1folder,f1instance)); /* read file one contents */
read_file_contents(FILE_TWO,AddPaths(f2folder,f2instance)); /* read file two contents */
if (DEBUG){ /* if debugging */
print_tables(); /* print hashtables */
} /* */
ProgressSetStatus("Copying Files"); /* update status */
ProgressUpdate(7,10); /* update progress */
rc = copy_files(f1instance,master_folder,merge_output); /* copy files into output folder */
if (IsError(rc)){ /* if they didn't copy */
MessageBox('x',"Cannot copy files to %s",merge_output); /* display error message */
return rc; /* return error code */
} /* */
write_instance(f1instance,master_folder,merge_output, bigger_file); /* write the instance file */
ProgressUpdate(10,10); /* update progress */
ProgressClose(); /* close progress */
MessageBox('i',"Files Merged Successfully, see folder %s", /* display message to user */
merge_output); /* display message to user */
CloseHandle(FileOneWindow); /* close window handle */
CloseHandle(FileTwoWindow); /* close window handle */
return ERROR_NONE; /* */
} /* */
/****************************************/
int write_instance(string instance,string master,string dest,int bigger){/* write the new instance file */
/****************************************/
int rc; /* return code */
int linetype; /* type of line being read */
int ix,rx; /* counters */
int num_rows; /* number of rows in a table */
int num_lines; /* number of lines in unit or context */
int big_index; /* index in the big arrays */
int small_index; /* index in the small arrays */
boolean written_contexts; /* true if contexts are written */
boolean written_units; /* true if contexts are written */
boolean written_facts; /* true if facts are written */
string hash; /* hash of a line */
string line; /* single line of a file */
string out_file; /* output file */
handle out; /* contents of output file */
string small_contexts[][]; /* contexts to use as source */
string small_units[][]; /* units to use as source */
string small_facts[]; /* facts to use as source */
string big_contexts[][]; /* contexts to use as dest */
string big_units[][]; /* units to use as dest */
string big_facts[]; /* facts to use as dest */
string src_ins; /* source instance filename */
handle src_ins_file; /* source instance file handle */
/* */
out = PoolCreate(); /* create pool */
src_ins = AddPaths(master,instance); /* source instance file */
src_ins_file = OpenFile(src_ins, FO_READ); /* open the instance file */
out_file = AddPaths(dest,instance); /* path to output file */
line = ReadLine(src_ins_file); /* read the first line from the file */
switch(bigger){ /* switch on the bigger file */
case FILE_ONE: /* if it's file one */
big_contexts = FileOneContexts; /* store contexts */
big_units = FileOneUnits; /* store units */
big_facts = FileOneFacts; /* store facts */
small_contexts = FileTwoContexts; /* store contexts */
small_units = FileTwoUnits; /* store units */
small_facts = FileTwoFacts; /* store facts */
break; /* break */
case FILE_TWO: /* if it's file two */
big_contexts = FileTwoContexts; /* store contexts */
big_units = FileTwoUnits; /* store units */
big_facts = FileTwoFacts; /* store facts */
small_contexts = FileOneContexts; /* store contexts */
small_units = FileOneUnits; /* store units */
small_facts = FileOneFacts; /* store facts */
break; /* break */
} /* */
while(line !=""){ /* while we have a next line */
linetype = get_line_type(line); /* get the type of line */
if(linetype==OTHER){ /* if this isn't the start of data */
PoolAppend(out,line); /* add line to output file */
PoolAppendNewLine(out); /* add line break */
} /* */
else{ /* if we're starting a merged data line */
debug_message("Reading line: "+line); /* debug message */
switch(linetype){ /* switch on type of line */
case CONTEXT: /* if we hit a context line */
if(written_contexts == false){ /* if we've not written contexts */
written_contexts = true; /* we're writing contexts now */
num_rows = ArrayGetAxisDepth(small_contexts); /* num rows in small contexts */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_contexts,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_contexts,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
num_lines = ArrayGetAxisDepth(small_contexts, /* get the number of columns */
AXIS_COL); /* get the number of columns */
for(rx=0;rx<num_lines;rx++){ /* for each column */
if(small_contexts[ix][rx]!=""){ /* make sure column has a value */
debug_message("Add context: "+small_contexts[ix][rx]);/* debug message */
PoolAppend(out,small_contexts[ix][rx]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
} /* */
} /* */
break; /* break */
case UNIT: /* if we've hit a unit line */
if(written_units == false){ /* if we've not written units */
written_units = true; /* we're writing units now */
num_rows = ArrayGetAxisDepth(small_units); /* num rows in small units */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_units,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_units,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
num_lines = ArrayGetAxisDepth(small_units,AXIS_COL);/* get the number of columns */
for(rx=0;rx<num_lines;rx++){ /* for each column */
if(small_units[ix][rx]!=""){ /* make sure column has a value */
debug_message("Add Unit: "+small_units[ix][rx]);/* debug message */
PoolAppend(out,small_units[ix][rx]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
} /* */
} /* */
break;
case FACT:
if(written_facts == false){ /* if we've not written facts */
written_facts = true; /* we're writing facts now */
num_rows = ArrayGetAxisDepth(small_facts); /* num rows in small facts */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_facts,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_facts,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
debug_message("Add fact: "+small_facts[ix]); /* debug_message */
PoolAppend(out,small_facts[ix]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
break;
}
PoolAppend(out,line); /* write line to output */
PoolAppendNewLine(out); /* add line break */
}
line = ReadLine(src_ins_file); /* read the line from our source */
} /* */
PoolWriteFile(out,out_file); /* write pool to output file */
CloseHandle(out); /* close pool */
CloseHandle(src_ins_file); /* close src file */
return ERROR_NONE; /* return */
} /* */
/****************************************/
int get_line_type(string line){ /* tests the type of line being read */
/****************************************/
if (FindInString(line,CONTEXT_START)>(-1)){ /* if it contains key value */
return CONTEXT; /* return true */
} /* */
if (FindInString(line,UNIT_START)>(-1)){ /* if it contains key value */
return UNIT; /* return true */
} /* */
if (FindInString(line,FACT_IDENTIFIER)>(-1)){ /* if it contains key value */
return FACT; /* return true */
} /* */
return OTHER; /* not merged data */
}
/****************************************/
int clear_folder(string path){ /* delete all files from given folder */
/****************************************/
string filenames[]; /* filenames to delete */
int num_files; /* number of files */
int rc; /* result */
int ix; /* counter */
/* */
path = AddPathDelimiter(path); /* ensure path ends in slash */
filenames = EnumerateFiles(path+"*.*"); /* get all files in folder */
num_files = ArrayGetAxisDepth(filenames); /* get the number of files in folder */
for (ix=0;ix<num_files;ix++){ /* for each file in folder */
rc = DeleteFile(AddPaths(path,filenames[ix])); /* delete it */
if (IsError(rc)){ /* if we couldn't delete it */
return rc; /* return the error code */
} /* */
} /* */
return ERROR_NONE; /* return no error */
}
/****************************************/
int copy_files(string instance, string source, string dest){ /* copies all non-instance files */
/****************************************/
string sourcefiles[]; /* filnenames in source */
string sourcefile; /* source filename */
string destfile; /* destination filename */
int ix; /* counter */
int num_files; /* number of files */
int rc; /* result */
/* */
source = AddPathDelimiter(source); /* ensure source ends in slash */
sourcefiles = EnumerateFiles(source+"*.*"); /* get all files in source */
num_files = ArrayGetAxisDepth(sourcefiles); /* get number of files */
for(ix=0;ix<num_files;ix++){ /* for each file */
if (sourcefiles[ix]!=instance){ /* if this isn't the instance file */
sourcefile = AddPaths(source,sourcefiles[ix]); /* source file to move */
destfile = AddPaths(dest,sourcefiles[ix]); /* destination of file */
rc = CopyFile(sourcefile,destfile); /* copy the file */
if (IsError(rc)){ /* if it didn't copy */
return rc; /* return error code */
} /* */
} /* */
} /* */
return ERROR_NONE; /* return */
}
/****************************************/
int read_file_contents(int file, string instance){ /* read the contents of the instance */
/****************************************/
handle instancefile; /* file we're reading */
int ix; /* counter */
int num_contexts; /* number of contexts */
int num_units; /* number of units */
int num_facts; /* number of facts */
string key; /* key to a table row */
string line; /* a line of the file we're reading */
string contexts[][]; /* table of contexts */
string units[][]; /* table of units */
string facts[]; /* array of facts */
/* */
instancefile = OpenFile(instance, FO_READ); /* open the file to read */
line = ReadLine(instancefile); /* read the next line of the file */
while (line!=""){ /* while we have a next line */
if (FindInString(line,CONTEXT_START)>(-1)){ /* if we are starting a context */
key = MD5CreateDigest(line,GetStringLength(line)); /* store key for context array */
contexts[key][0]=line; /* store first line of context in array */
ix = 1; /* initialize ix as row 1 of table */
while (line!="" && FindInString(line,CONTEXT_END)<0){ /* loop until we end context */
line = ReadLine(instancefile); /* read the next line */
contexts[key][ix] = line; /* store next line of context */
ix++; /* increment counter */
} /* */
} /* */
if (FindInString(line,UNIT_START)>(-1)){ /* if we are starting a unit */
key = MD5CreateDigest(line,GetStringLength(line)); /* store key for context array */
units[key][0]=line; /* store first line of unit in array */
ix = 1; /* initialize ix as row 1 of table */
while (line!="" && FindInString(line,UNIT_END)<0){ /* loop until we end unit */
line = ReadLine(instancefile); /* read the next line */
units[key][ix] = line; /* store next line of unit */
ix++; /* increment counter */
} /* */
} /* */
if (FindInString(line,FACT_IDENTIFIER)>(-1)){ /* if we have a fact */
facts[MD5CreateDigest(line,GetStringLength(line))]=line; /* store the fact in our table */
} /* */
line = ReadLine(instancefile); /* read the next line of the file */
} /* */
num_contexts = ArrayGetAxisDepth(contexts); /* get number of contexts */
num_units = ArrayGetAxisDepth(units); /* get number of units */
num_facts = ArrayGetAxisDepth(facts); /* get number of facts */
debug_message(FormatString("File: %d\r\nContexts: %d\r\nUnits: "+ /* display message */
"%d\r\nFacts: %d",file,num_contexts,num_units,num_facts)); /* display message */
CloseHandle(instancefile); /* close the open file */
switch(file){ /* switch on what file we're reading */
case FILE_ONE: /* if it's file one */
FileOneContexts = contexts; /* store contexts */
FileOneUnits = units; /* store units */
FileOneFacts = facts; /* store facts */
break; /* break */
case FILE_TWO: /* if it's file two */
FileTwoContexts = contexts; /* store contexts */
FileTwoUnits = units; /* store units */
FileTwoFacts = facts; /* store facts */
break; /* break */
} /* */
return ERROR_NONE; /* */
} /* */
/****************************************/
int compare_filesizes(string f1, string f2){ /* compare two file sizes */
/****************************************/
qword f1size; /* file 1 size */
qword f2size; /* file 2 size */
int sizedif; /* difference in sizes */
/* */
debug_message("Comparing "+f1+" to "+f2); /* debug message */
debug_message(FormatString("Name size dif: %d",namesize_dif)); /* debug message */
f1size = GetFileSize(f1); /* get size of file 1 */
debug_message(FormatString("F1 Size: %d",f1size)); /* debug message */
f2size = GetFileSize(f2); /* get size of file 2 */
debug_message(FormatString("F2 Size: %d",f2size)); /* debug message */
sizedif = f1size-f2size; /* get difference in sizes */
if ((Absolute(sizedif) - namesize_dif) == 0){ /* if files are same sized */
debug_message("Files are equal in size."); /* debug message */
return OTHER; /* return other */
} /* */
else{ /* if they are not the same size */
if (f1size>f2size){ /* if file one is bigger */
debug_message("File One is bigger."); /* add debug message */
return FILE_ONE; /* return file one */
} /* otherwise */
debug_message("File Two is bigger."); /* debug message */
return FILE_TWO; /* return file two */
} /* */
} /* */
/****************************************/
int compare_xbrl(string instance,string f1folder, string f2folder){ /* test if 2 XBRL files can be merged */
/****************************************/
/* */
int ix; /* loop counter */
int num_files; /* number of files in folder */
int sizeres; /* result of getting size dif */
string f1filepath; /* path to a file in folder 1 */
string f2filepath; /* path to a file in folder 2 */
string f1files[]; /* files in folders one */
/* */
f1folder = AddPathDelimiter(f1folder); /* ensure path ends in a slash */
f1files = EnumerateFiles(f1folder+"*.*"); /* get filenames in folder one */
/* */
num_files = ArrayGetAxisDepth(f1files); /* number of files in folder one */
for (ix=0;ix<num_files;ix++){ /* for each file in folder one */
f1filepath = AddPaths(f1folder,f1files[ix]); /* get path to file in folder one */
f2filepath = AddPaths(f2folder,f1files[ix]); /* get path to file in folder two */
if(IsFile(f2filepath)==false){ /* if this file doesn't exist in f2 */
SetLastError(ERROR_EXIT,"File "+f2filepath+" does not exist"); /* set error message */
return ERROR_EXIT; /* return with error */
} /* */
sizeres = compare_filesizes(f1filepath,f2filepath); /* get the file size result */
if(sizeres!=OTHER){ /* if the sizes don't match up */
if (f1files[ix]!=instance){ /* and this isn't the instance file */
bigger_file = sizeres; /* store the bigger file */
SetLastError(ERROR_WARN,"File "+f1files[ix]+" does not "+ /* set error */
"match." ); /* set error */
return ERROR_WARN; /* return error */
} /* */
} /* */
} /* */
return ERROR_NONE; /* return without error */
} /* */
/****************************************/
string export_file(string foldersuffix,string file, handle window){ /* export the XFR file */
/****************************************/
string path; /* path to output file */
string response; /* response from export */
string filenames[]; /* filenames */
string cmd; /* command to export */
int ix; /* counter */
int num_files; /* number of files */
/* */
path = get_export_folder(file,foldersuffix); /* get folder to export to */
if (path == ""){ /* if the export folder is blank */
return ""; /* return an error */
} /* */
path = AddPathDelimiter(path); /* ensure path ends in slash */
cmd = "NoQuery: TRUE; Path: " + path; /* generate command string */
if (IsWindowHandleValid(window)){ /* check if file is already open */
ActivateEditWindow(window); /* activate edit window */
RunMenuFunction("XBRL_EXPORT",cmd); /* export the file */
} /* */
else{ /* */
RunMenuFunction("FILE_OPEN","Filename:"+file); /* open file */
RunMenuFunction("XBRL_EXPORT",cmd); /* export the file */
} /* */
response = GetMenuFunctionResponse(); /* response from the export */
return GetParameter(response,"Instance"); /* return the name of the instance file */
}
/****************************************/
string get_export_folder(string file, string foldersuffix){ /* build the path to the output folder */
/****************************************/
int rc; /* return code */
string folder; /* folder for XFR file */
string newfolder; /* new folder for exporting to */
/* */
folder = GetFilePath(file); /* get folder */
newfolder = EXPORT_FOLDER_PREFIX+foldersuffix; /* build name of new folder */
newfolder = AddPaths(folder,newfolder); /* build full path to newfolder */
if (IsFolder(newfolder)==false){ /* if folder doesn't exist */
rc = CreateFolder(newfolder); /* try to create the folder */
if (IsError(rc)){ /* did it create OK? */
return ""; /* return a blank string */
} /* */
} /* */
rc = clear_folder(newfolder); /* clear the export folder */
if (IsError(rc)){ /* if we couldn't clear the export */
SetLastError(rc,"Cannot delete files in folder "+newfolder); /* set error */
return ""; /* return */
} /* */
return newfolder; /* return the new folder path */
}
/****************************************/
int main(){ /* main function */
/****************************************/
string s1; /* General */
/* */
s1 = GetScriptParent(); /* Get the parent */
if (s1 == "LegatoIDE") { /* Is run from the IDE (debug) */
setup(); /* run setup */
run(0,"preprocess"); /* run as though hooked */
} /* end IDE run */
return ERROR_NONE; /* */
} /* */
/****************************************/
int merge_load(){ /* Setup Action */
/****************************************/
string file_one,file_two; /* old file paths */
/* */
file_one = GetSetting("XBRLMerge","File One"); /* get path to file one */
file_two = GetSetting("XBRLMerge","File Two"); /* get path to file two */
/* */
EditSetText(XBRL_ONE_TEXT,file_one); /* set edit text */
EditSetText(XBRL_TWO_TEXT,file_two); /* set edit text */
return ERROR_NONE; /* */
}
/****************************************/
int merge_action(int c_id, int c_ac) { /* Control Action */
/****************************************/
string s1; /* General */
/* */
/* ** Control Actions */
/* * Browse for XML 1 */
if (c_id == XBRL_RESET){ /* if resetting */
EditSetText(XBRL_ONE_TEXT,""); /* reset text of box */
PutSetting("XBRLMerge","File One",""); /* reset setting file */
EditSetText(XBRL_TWO_TEXT,""); /* reset text of box */
PutSetting("XBRLMerge","File Two",""); /* reset setting file */
} /* */
/* */
if (c_id == XBRL_ONE_BROWSE) { /* Control ID (button) */
s1 = EditGetText(XBRL_ONE_TEXT); /* Get the current path */
s1 = BrowseOpenFile("Select First XBRL File","*.xfr|*.xfr", s1); /* Browse for the folder */
if (s1 != "") { /* Returned a value (OK) */
EditSetText(XBRL_ONE_TEXT, s1); /* Get the current path */
PutSetting("XBRLMerge","File One",s1); /* store setting for later */
} /* end has string */
return ERROR_NONE; /* Done */
} /* end browse */
/* * Browse for XML 2 */
if (c_id == XBRL_TWO_BROWSE) { /* Control ID (button) */
s1 = EditGetText(XBRL_TWO_TEXT); /* Get the current path */
if (s1 == "") { /* Empty, pick up source */
s1 = EditGetText(XBRL_ONE_TEXT); /* Get the current path */
} /* end has string */
s1 = BrowseOpenFile("Select Second XBRL File","*.xfr|*.xfr",s1); /* Browse for the folder */
if (s1 != "") { /* Returned a value (OK) */
EditSetText(XML_TWO_TEXT, s1); /* Get the current path */
PutSetting("XBRLMerge","File Two",s1); /* store setting for later */
} /* end has string */
return ERROR_NONE; /* Done */
} /* end browse */
return ERROR_NONE; /* Exit no error */
} /* end routine */
/****************************************/
int merge_validate(){ /* Validate Action */
/****************************************/
int valid_one,valid_two; /* validations of files */
string file_one,file_two; /* file paths */
/* */
file_one = EditGetText(XBRL_ONE_TEXT); /* get path to file one */
file_two = EditGetText(XBRL_TWO_TEXT); /* get path to file two */
if (file_one == "" || file_two == ""){ /* if either file is blank */
MessageBox('x',"Two files must be selected."); /* make sure two files are selected */
return ERROR_EXIT; /* exit with error */
} /* */
if (file_one == file_two){ /* test if same file */
MessageBox('x',"You cannot merge a file into itself."); /* display error message */
return ERROR_EXIT; /* return with error */
}
valid_one = validate_file(file_one,"One"); /* validate file_one */
if (IsWindowHandleValid(FileTwoWindow)){ /* if our validate set a file handle */
FileOneWindow = FileTwoWindow; /* store handle as file one */
FileTwoWindow = NULL_HANDLE; /* close file two window */
} /* */
valid_two = validate_file(file_two,"Two"); /* validate file_two */
if (valid_one==valid_two && valid_two==ERROR_NONE){ /* if both validates returned ERROR_NONE*/
return ERROR_NONE; /* return no error */
} /* */
return ERROR_EXIT; /* exit with an error */
}
/****************************************/
int validate_file(string file, string display, handle file_handle){ /* validate an individual file */
/****************************************/
int rc; /* return code from our file */
int depth; /* number of windows open */
int ix; /* array index counter */
/* */
depth = ArrayGetAxisDepth(edit_windows); /* get number of windows open */
if (IsFile(file)){ /* check if file one is a file */
if (CanAccessFile(file,FO_WRITE)){ /* check if we can write to the file */
if (MakeLowerCase(GetExtension(file))==".xfr"){ /* make sure file ends in .xfr */
return ERROR_NONE; /* return no error */
} /* */
else{ /* if file doesn't end in .xfr */
MessageBox('x',"File "+display+" Must be an XFR file."); /* display error message */
} /* */
} /* */
else{ /* if we cannot write to file */
for (ix = 0; ix<depth; ix++){ /* scan all open windows */
if (edit_windows[ix]["Filename"] == file){ /* if the file is already open */
FileTwoWindow=MakeHandle(edit_windows[ix]["ClientHandle"]); /* get the handle to it */
rc = GetLastError(); /* check if we got an error */
if (IsError(rc)){ /* if we have an error */
MessageBox('x',"Cannot create handle, error %0x",rc); /* display error message */
} /* */
else{ /* if we don't have an error */
return ERROR_NONE; /* the file is already open, return */
} /* */
} /* */
} /* */
MessageBox('x',"Cannot open File "+display); /* give error message */
} /* */
} /* */
else{ /* if we cannot open file one */
MessageBox('x',"File "+display+" does not exist."); /* give error message */
} /* if we haven't exited yet */
return ERROR_EXIT; /* return an error */
} /* */
/****************************************/
int debug_message(message){ /* add a message to the debug log */
/****************************************/
if (DEBUG){ /* if debugging */
if(GetStringLength(message)>DEBUG_MSG_MAX){ /* if message is longer than 100 */
message = GetStringSegment(message,0,DEBUG_MSG_MAX); /* print out first 100 chars */
} /* */
AddMessage(message); /* log the message */
} /* */
return ERROR_NONE; /* */
} /* */
/****************************************/
int print_tables(){ /* debug function to print tables */
/****************************************/
int num_rows; /* num rows */
int ix; /* counter */
/* */
debug_message("Unit Table 1:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileOneUnits); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash: "+ArrayGetKeyName(FileOneUnits,ix)); /* add a debug message */
debug_message("Unit: "+FileOneUnits[ix][0]); /* print out info */
} /* */
debug_message("Unit Table 2:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileTwoUnits); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash: "+ArrayGetKeyName(FileTwoUnits,ix)); /* add a debug message */
debug_message("Unit: "+FileTwoUnits[ix][0]); /* print out info */
} /* */
debug_message("*****************************"); /* add spacer */
debug_message("Context Table 1:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileOneContexts); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash : "+ArrayGetKeyName(FileOneContexts,ix)); /* add a debug message */
debug_message("Context: "+FileOneContexts[ix][0]); /* print out info */
} /* */
debug_message("Context Table 2:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileTwoContexts); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash : "+ArrayGetKeyName(FileTwoContexts,ix)); /* add a debug message */
debug_message("Context: "+FileTwoContexts[ix][0]); /* print out info */
} /* */
debug_message("*****************************"); /* add spacer */
debug_message("Fact Table 1:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileOneFacts); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash: "+ArrayGetKeyName(FileOneFacts,ix)); /* add a debug message */
debug_message("Fact: "+FileOneFacts[ix]); /* print out info */
} /* */
debug_message("Fact Table 2:"); /* add a message */
num_rows = ArrayGetAxisDepth(FileTwoFacts); /* get num rows */
for(ix=0;ix<num_rows;ix++){ /* for each row */
debug_message("Hash: "+ArrayGetKeyName(FileTwoFacts,ix)); /* add a debug message */
debug_message("Fact: "+FileTwoFacts[ix]); /* print out info */
} /* */
return ERROR_NONE; /* */
} /* */
/****************************************/
int merge_ok(){ /* OK Action */
/****************************************/
int f1namesize,f2namesize; /* sizes of the names of xfr files */
string f1,f2; /* file names chosen by user */
/* */
FileOne = EditGetText(XBRL_ONE_TEXT); /* get path to file one */
FileTwo = EditGetText(XBRL_TWO_TEXT); /* get path to file two */
/* */
f1 = GetFilename(FileOne); /* get the name of the first xfr file */
f2 = GetFilename(FileTwo); /* get the name of the second xfr file */
f1namesize = GetStringLength(EncodeURIComponent(f1)); /* size of file 1 name */
f2namesize = GetStringLength(EncodeURIComponent(f2)); /* size of file 2 name */
namesize_dif = Absolute(f1namesize-f2namesize); /* get size of dif in name sizes */
return ERROR_NONE; /* return that the user pressed OK */
}
We only have three new defines this week: CONTEXT, UNIT, and FACT. These are going to be output from the get_line_type function. It really helps readability to use defines like this for constant values that represent something, so you can see what the intent of that code is. Otherwise, checking if a value equals 4, for example, later on won’t mean much to anyone reviewing the code for the first time.
#define CONTEXT 3
#define UNIT 4
#define FACT 5
Our first new function this week is copy_files. This is a pretty basic one. Its purpose is to move all the files (except the instance file) from any folder into the destination folder. Since the instance file isn’t supposed to be moved, we need to take its name and path as a parameter, as well as the source and destination files. Since the EnumerateFiles SDK function needs to take a path ending in *.*, the first thing we need to do is ensure our source path ends in a path delimiter by using the AddPathDelimiter function on it. We can then pass it to our EnumerateFilesFunction (concatenated with the *.*) to get a list of all files in the source folder. Using a for loop, we iterate over every file in the source folder. If the file is not equal to our instance file, then we can build a path for our source file and destination file by using the AddPaths function and send both of those as parameters to the CopyFile function. Each time the CopyFile function is called, we need to check if the return code is an error with the IsError function. If it’s an error, we need to return that error so it can be handled. Otherwise, we can continue until all files are processed and return without error.
/****************************************/
int copy_files(string instance, string source, string dest){ /* copies all non-instance files */
/****************************************/
string sourcefiles[]; /* filnenames in source */
string sourcefile; /* source filename */
string destfile; /* destination filename */
int ix; /* counter */
int num_files; /* number of files */
int rc; /* result */
/* */
source = AddPathDelimiter(source); /* ensure source ends in slash */
sourcefiles = EnumerateFiles(source+"*.*"); /* get all files in source */
num_files = ArrayGetAxisDepth(sourcefiles); /* get number of files */
for(ix=0;ix<num_files;ix++){ /* for each file */
if (sourcefiles[ix]!=instance){ /* if this isn't the instance file */
sourcefile = AddPaths(source,sourcefiles[ix]); /* source file to move */
destfile = AddPaths(dest,sourcefiles[ix]); /* destination of file */
rc = CopyFile(sourcefile,destfile); /* copy the file */
if (IsError(rc)){ /* if it didn't copy */
return rc; /* return error code */
} /* */
} /* */
} /* */
return ERROR_NONE; /* return */
}
This next function, get_line_type, is probably the simplest function in this entire script. Its only purpose is to examine a given string, and if it’s the start of a fact, context, or unit, return the appropriate define. To do that, we have three if statements, each checking the result of a FindInString function. For the first statement, we look for the CONTEXT_START define value using the FindInString function, and if the function returns anything greater than negative one, it means that the string contains that define. That then means it’s the start of a context, so we can return our CONTEXT define. We repeat that same logic in the next two statements, checking for UNIT_START or FACT_IDENTIFIER, and returning UNIT or FACT respectively. If we haven’t returned anything by the end of the last if statement, we can assume the given line isn’t the start of any of those three line types, so we can return OTHER.
/****************************************/
int get_line_type(string line){ /* tests the type of line being read */
/****************************************/
if (FindInString(line,CONTEXT_START)>(-1)){ /* if it contains key value */
return CONTEXT; /* return true */
} /* */
if (FindInString(line,UNIT_START)>(-1)){ /* if it contains key value */
return UNIT; /* return true */
} /* */
if (FindInString(line,FACT_IDENTIFIER)>(-1)){ /* if it contains key value */
return FACT; /* return true */
} /* */
return OTHER; /* not merged data */
}
Now that we have the simplest function out of the way, let’s take a look at the most complicated one: our write_instance function. The general purpose of this function is to put everything from the smaller instance file into the bigger one. To do that, we open the bigger file, scan it, and check the line type of every line. When we hit the first context line, we write out all of the contexts that exist in the smaller one that do not exist in the bigger one. Then we go back to scanning the file. Once we hit the first unit, we do the exact same for units, writing out all the ones that do not exist in the bigger file. We can then repeat that exact same step when we hit the facts. This does mean that if our bigger file doesn’t have any contexts, facts, or units, nothing will be merged. However, this isn’t something that should really ever happen. If one file doesn’t have any data, then there’s no point in merging them in the first place really.
To start with, we need to create our output string pool, called out, using the PoolCreate function. A string pool functions a lot like a string, but it offers much better performance if we’re going to be constantly adding to it one line at a time. Using a regular string works too, but because the size of the string would constantly have to be reallocated by Legato while running, the script’s performance would take a massive hit. Practically speaking, it takes 5-10 times longer using a string instead of a string pool in this particular script for test files we had. Next, we can get the path to our master instance file by using the AddPaths function to combine our “master” and “instance” parameters, which should be the master folder (the bigger one) and the instance filename. We can build the destination of our output by using the AddPaths function as well, just on the “dest” and “instance” parameters. After that, we grab a handle to our master instance file with the OpenFile function and read the first line with the ReadLine function.
Before we go any further with this, we need to determine if this function is going to use the FileOne data structures (for the facts, contexts, etc) for the bigger file or the FileTwo data structures for the bigger file. To do this, we can use a switch statement on the bigger parameter. If FILE_ONE is passed as the “bigger” parameter, we know it’s the bigger file, so it gets stored in the data structures in this function with the “big_” prefix. The data structures with the “small_” prefix will hold the data for file two. The opposite happens if FILE_TWO is the bigger of the two files.
/****************************************/
int write_instance(string instance,string master,string dest,int bigger){/* write the new instance file */
/****************************************/
int rc; /* return code */
int linetype; /* type of line being read */
int ix,rx; /* counters */
int num_rows; /* number of rows in a table */
int num_lines; /* number of lines in unit or context */
int big_index; /* index in the big arrays */
int small_index; /* index in the small arrays */
boolean written_contexts; /* true if contexts are written */
boolean written_units; /* true if contexts are written */
boolean written_facts; /* true if facts are written */
string hash; /* hash of a line */
string line; /* single line of a file */
string out_file; /* output file */
handle out; /* contents of output file */
string small_contexts[][]; /* contexts to use as source */
string small_units[][]; /* units to use as source */
string small_facts[]; /* facts to use as source */
string big_contexts[][]; /* contexts to use as dest */
string big_units[][]; /* units to use as dest */
string big_facts[]; /* facts to use as dest */
string src_ins; /* source instance filename */
handle src_ins_file; /* source instance file handle */
/* */
out = PoolCreate(); /* create pool */
src_ins = AddPaths(master,instance); /* source instance file */
src_ins_file = OpenFile(src_ins, FO_READ); /* open the instance file */
out_file = AddPaths(dest,instance); /* path to output file */
line = ReadLine(src_ins_file); /* read the first line from the file */
switch(bigger){ /* switch on the bigger file */
case FILE_ONE: /* if it's file one */
big_contexts = FileOneContexts; /* store contexts */
big_units = FileOneUnits; /* store units */
big_facts = FileOneFacts; /* store facts */
small_contexts = FileTwoContexts; /* store contexts */
small_units = FileTwoUnits; /* store units */
small_facts = FileTwoFacts; /* store facts */
break; /* break */
case FILE_TWO: /* if it's file two */
big_contexts = FileTwoContexts; /* store contexts */
big_units = FileTwoUnits; /* store units */
big_facts = FileTwoFacts; /* store facts */
small_contexts = FileOneContexts; /* store contexts */
small_units = FileOneUnits; /* store units */
small_facts = FileOneFacts; /* store facts */
break; /* break */
} /* */
Our data structures are loaded, our file is open, the output is ready, it’s time to actually get to work. While our line isn’t blank, we can get the line type of our current line with our get_line_type function. If it returns OTHER, then it’s not an important line, so we can just use the PoolAppend function to stick it onto the end of our string pool. We want every line to go onto its own line in the output, so after we append a new line, we need to use the PoolAppendNewLine function to tack on a line ending. If the line type isn’t OTHER though, we get to do some more interesting things. Firstly, we should put out a debug message. As we discussed last week, it’s important to sprinkle debug messages in the script so we can see exactly how things are working in the output. Then we can perform a switch operation on the type of line. If it’s a CONTEXT, we need to check if we’ve already written contexts out. The variable “written_contexts” should start as false, so this section of code will run the first time through. By immediately setting it to true in the next line, we ensure that this fragment of code is only ever run once.
After this, we can use a for loop to iterate over each of the contexts in our “small_contexts” structure. For each of them, we want to get the hash used as a key value for that context first. To do that, we use the ArrayGetKeyName function. With that hash value, we can test if the “big_contexts” structure contains any contexts that match this hash with the ArrayFindKeyName function. If that function returns less than zero, it means the hash doesn’t exist in the other structure, so we can proceed to write out the context information. To do this, we want to use a for loop to iterate over every column in that row, and for each, check if data actually exists for that table row/column combination. The ArrayGetAxisDepth function returns the total number of columns in the table. Legato two-dimensional arrays (and by extension, our hash tables) are actual tables, and as such every row has the same number of columns whether the “cell” has contents or not. Therefore, some positions in the array may be empty. If this particular table “cell” has a value, then we can put out a debug message containing that information. Then we use the PoolAppend and PoolAppendNewLine functions to add this context to our string pool.
while(line !=""){ /* while we have a next line */
linetype = get_line_type(line); /* get the type of line */
if(linetype==OTHER){ /* if this isn't the start of data */
PoolAppend(out,line); /* add line to output file */
PoolAppendNewLine(out); /* add line break */
} /* */
else{ /* if we're starting a merged data line */
debug_message("Reading line: "+line); /* debug message */
switch(linetype){ /* switch on type of line */
case CONTEXT: /* if we hit a context line */
if(written_contexts == false){ /* if we've not written contexts */
written_contexts = true; /* we're writing contexts now */
num_rows = ArrayGetAxisDepth(small_contexts); /* num rows in small contexts */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_contexts,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_contexts,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
num_lines = ArrayGetAxisDepth(small_contexts, /* get the number of columns */
AXIS_COL); /* get the number of columns */
for(rx=0;rx<num_lines;rx++){ /* for each column */
if(small_contexts[ix][rx]!=""){ /* make sure column has a value */
debug_message("Add context: "+small_contexts[ix][rx]);/* debug message */
PoolAppend(out,small_contexts[ix][rx]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
} /* */
} /* */
break; /* break */
We’re going to use the exact same logic here on our units. If we encounter a unit, we iterate over all units in the smaller unit structure, check to see if that unit exists in the bigger unit structure by comparing hashes from our hash tables, and if not, we append all columns in that unit row to our string pool. The same logic also applies to our fact values when we hit the first fact. Facts are even simpler, because there is no need to iterate over all columns in the fact table as it has only one column. Once we finish our switch statement to write out the data from the smaller file, we need to write out the original line read from the bigger file, so we use the PoolAppend and PoolAppendNewLine functions again. Then we can read the next line and start all over.
case UNIT: /* if we've hit a unit line */
if(written_units == false){ /* if we've not written units */
written_units = true; /* we're writing units now */
num_rows = ArrayGetAxisDepth(small_units); /* num rows in small units */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_units,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_units,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
num_lines = ArrayGetAxisDepth(small_units,AXIS_COL);/* get the number of columns */
for(rx=0;rx<num_lines;rx++){ /* for each column */
if(small_units[ix][rx]!=""){ /* make sure column has a value */
debug_message("Add Unit: "+small_units[ix][rx]);/* debug message */
PoolAppend(out,small_units[ix][rx]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
} /* */
} /* */
break; /* */
case FACT: /* */
if(written_facts == false){ /* if we've not written facts */
written_facts = true; /* we're writing facts now */
num_rows = ArrayGetAxisDepth(small_facts); /* num rows in small facts */
for(ix = 0; ix< num_rows;ix++){ /* for each row */
hash = ArrayGetKeyName(small_facts,ix,AXIS_ROW); /* get hash for this row */
if(ArrayFindKeyName(big_facts,hash,AXIS_ROW)<0){ /* if this doesn't exist in our big file*/
debug_message("Add fact: "+small_facts[ix]); /* debug_message */
PoolAppend(out,small_facts[ix]); /* add line to output */
PoolAppendNewLine(out); /* add line break */
} /* */
} /* */
} /* */
break; /* */
} /* */
PoolAppend(out,line); /* write line to output */
PoolAppendNewLine(out); /* add line break */
}
line = ReadLine(src_ins_file); /* read the line from our source */
} /* */
After we have our string pool filled, we can use the PoolWriteFile function to write it to our output destination. After that, the CloseHandle function is used to close all open handles. Finally we return without error.
PoolWriteFile(out,out_file); /* write pool to output file */
CloseHandle(out); /* close pool */
CloseHandle(src_ins_file); /* close src file */
return ERROR_NONE; /* return */
} /* */
The run function is almost identical to what it was before. Only the changed section is shown below. After the ProgressUpdate function runs, we need to call our copy_files function. This takes all of the files except the instance file from our bigger folder and puts them into our output folder. We then check the error code, and if it was an error, we display an error message and return. Otherwise, we can execute the write_instance function, which actually writes out our merged instance file. Finally, we can then close our progress window, display a message box with a success message, and close our open handles.
read_file_contents(FILE_ONE,AddPaths(f1folder,f1instance)); /* read file one contents */
read_file_contents(FILE_TWO,AddPaths(f2folder,f2instance)); /* read file two contents */
if (DEBUG){ /* if debugging */
print_tables(); /* print hashtables */
} /* */
ProgressSetStatus("Copying Files"); /* update status */
ProgressUpdate(7,10); /* update progress */
rc = copy_files(f1instance,master_folder,merge_output); /* copy files into output folder */
if (IsError(rc)){ /* if they didn't copy */
MessageBox('x',"Cannot copy files to %s",merge_output); /* display error message */
return rc; /* return error code */
} /* */
write_instance(f1instance,master_folder,merge_output, bigger_file); /* write the instance file */
ProgressUpdate(10,10); /* update progress */
ProgressClose(); /* close progress */
MessageBox('i',"Files Merged Successfully, see folder %s", /* display message to user */
merge_output); /* display message to user */
CloseHandle(FileOneWindow); /* close window handle */
CloseHandle(FileTwoWindow); /* close window handle */
return ERROR_NONE; /* */
} /* */
We finally have a fully functioning script! This is an example of a medium length script that can enhance GoFiler’s native functionality. The script could definitely be expanded to cover more than just merging an instance file. For example, it could merge lab files as well or any of the linkbase files. This script just assumes that everything in those linkbases exists in the bigger of the two files, which, for the user who wants to combine two very similar files, should be true. Expanding the functionality further would probably require the use of the SGML parser to read files and extensive knowledge about XBRL to devise the appropriate algorithms to combine the linkbases.
UI improvements could also be made. For example, if your current active window in GoFiler is already an XFR file, it could automatically fill in the File One field in the file selection prompt with that file name and path, assuming you want to merge another file into it. It could also automatically proof and test file the combined merged file with the SEC to ensure integrity.
As with any software project, however, be aware of creeping elegance. This script gets the job done in ~750 lines of code, which, while not insignificant, isn’t a very involved project. Adding more features to it, or more options, will greatly increase the complexity of the script and may not provide enough functionality to justify the increased development time. If it takes a week to write a script that will be used once a year and saves 20 minutes, then it may be fun to play with it and expand it, but it might not be worth the time investment.
 Steven Horowitz has been working for Novaworks for over five years as a technical expert with a focus on EDGAR HTML and XBRL. Since the creation of the Legato language in 2015, Steven has been developing scripts to improve the GoFiler user experience. He is currently working toward a Bachelor of Sciences in Software Engineering at RIT and MCC. |
Additional Resources
Legato Script Developers LinkedIn Group
Primer: An Introduction to Legato
Quicksearch
Categories
Calendar
Feeds
