So far, every script we’ve explored has been a hook of some sort. They attach themselves to new or existing menu icons or are triggered by other functions being run. What if you want something to run on application start? Legato supports this through the Application Initialize script. On program start, GoFiler will always run the ApplicationStartup.ls script. This script file is the one that’s responsible for running every other script in the extensions folder. This allows users to create their own custom startup routines that are run before literally anything else.
Friday, June 16. 2017
LDC #39: Application Initialize Script
Today, our example will be the ApplicationInitialize.ls file, configured to check to make sure the user is running the most up to date CUSIP list for 13-F validation. This is going to be included in future versions of GoFiler Complete and Go13, but it makes a good example blog post as well.
The script:
//
// GoFiler Complete - Initialize Application
// ------------------------------------
//
// Runs on opening application. Currently checks 13F CUSIP list.
//
// Revised 06/13/2017 Initial
//
//
// (c) 2017 Novaworks, LLC. All rights reserved.
#define CUSIP_LIST_BASE "https://www.sec.gov/divisions/investment/13f/13flistYYYYqQQQQ.pdf"
#define TEMP_FILENAME "temp_cusips.pdf"
#define CUSIP_CSV "cusips.csv"
#define LAST_ATTEMPT "Last CUSIP Download Attempt"
#define ONE_DAY 864000000000
int check_cusip_list (); /* check the current CUSIP list */
/****************************************/
int main(){ /* runs on startup */
/****************************************/
check_cusip_list(); /* run the check cusip function */
return ERROR_NONE; /* return without error */
} /* */
/****************************************/
int check_cusip_list(){ /* check that the cusip list is current */
/****************************************/
qword time; /* current time */
qword last_attempt; /* last attempted download */
int rc; /* result */
string time_s; /* time as a string */
string product; /* name of product */
string appdata; /* location of appdata folder */
string settings; /* gofiler's settings file */
string cusip_file; /* cusip file */
string year; /* year */
string month; /* month of the year */
string quarter; /* quarter of year as a string */
string old_cusip_csv; /* the old cusips CSV file */
string cusip_loc; /* location of cusip file to try */
string library_loc; /* library location on machine */
string tempfile; /* temporary cusip file */
string current_list; /* current cusip list */
/* */
time = GetLocalTime(); /* get the local time */
month = MakeLowerCase(FormatDate(time,"M")); /* get current month */
if (month == "jan" || month == "feb" || month == "mar"){ /* if q1 */
quarter = "4"; /* use previous quarter */
} /* */
if (month == "apr" || month == "may" || month == "jun"){ /* if q2 */
quarter = "1"; /* use previous quarter */
} /* */
if (month == "jul" || month == "aug" || month == "sep"){ /* if q3 */
quarter = "2"; /* use previous quarter */
} /* */
if (month == "oct" || month == "nov" || month == "dec"){ /* if q4 */
quarter = "3"; /* use previous quarter */
} /* */
year = FormatDate(time,"o"); /* get year */
if (quarter == "4"){ /* if we want Q4's list */
year = FormatString("%d",TextToInteger(year)-1); /* subtract a year */
} /* */
cusip_loc = ReplaceInString(CUSIP_LIST_BASE,"YYYY",year); /* add year to cusip string */
cusip_loc = ReplaceInString(cusip_loc,"QQQQ",quarter); /* add quarter to cusip string */
appdata = GetApplicationDataFolder(); /* get the appdata folder */
product = GetApplicationName(); /* get the application name */
settings = product + " Settings.ini"; /* get name of settings file */
settings = AddPaths(appdata,settings); /* get path to settings */
current_list = GetSetting(settings,"Form 13F Options","PDF CUSIPs");/* get current cusip list */
library_loc = GetSetting(settings,"EDS","Library"); /* get library location */
old_cusip_csv = AddPaths(library_loc,CUSIP_CSV); /* get path to old cusip csv file */
if (current_list == cusip_loc){ /* if we have the most up to date list */
return ERROR_NONE; /* return without error */
} /* */
last_attempt = GetSetting(settings,"Form 13F Options",LAST_ATTEMPT);/* get last attempt to dl file */
if ((time - last_attempt)<ONE_DAY){ /* if less than one day has passed */
return ERROR_NONE; /* return without trying again */
} /* */
tempfile = AddPaths(GetTempFileFolder(),TEMP_FILENAME); /* get path to temp file */
rc = HTTPGetFile(cusip_loc,tempfile); /* get temp file */
DeleteFile(tempfile); /* remove temp file */
if (rc!=200){ /* if we failed to get the file */
PutSetting(settings,"Form 13F Options",LAST_ATTEMPT,time_s); /* store last attempted dl time */
return ERROR_NONE; /* return without error */
} /* */
PutSetting(settings,"Form 13F Options","PDF CUSIPs",cusip_loc); /* store new cusip list */
DeleteFile(old_cusip_csv); /* delete the old CUSIP CSV file */
LoadPreferences(); /* force preferences to reload */
return ERROR_NONE; /* */
} /* */
The first thing to notice about this script is that it’s much simpler in structure than other ones in the past. It simply has two functions: main and check_cusip_list. It really doesn’t need anything other than main, but if other initialization functions need to be added later to the ApplicationInitialize.ls file, it helps to have the script organized into different functions for each action that needs to be performed. There is no need for a setup function because the main function is automatically run every time the application starts up.
#define CUSIP_LIST_BASE "https://www.sec.gov/divisions/investment/13f/13flistYYYYqQQQQ.pdf" #define TEMP_FILENAME "temp_cusips.pdf" #define CUSIP_CSV "cusips.csv" #define LAST_ATTEMPT "Last CUSIP Download Attempt" #define ONE_DAY 864000000000
The script uses five different defines. CUSIP_LIST_BASE is the template location for CUSIP lists on the SEC’s website. The YYYY value in the string needs to be replaced with the correct year, and the QQQQ value needs to be replaced with the correct quarter. The TEMP_FILENAME is the name of the file we will try to download off the SEC’s site. It’s temporary because we only need to download it to check if it is available, after which it can be deleted. The CUSIP_CSV define is GoFiler’s internal CUSIP library file. After we update the list, we need to delete this to force GoFiler to rebuild it with the new list. LAST_ATTEMPT stores the last time a CUSIP download was attempted and failed to ensure we don’t spam the SEC’s website every time GoFiler opens to request a new CUSIP list that doesn’t exist. ONE_DAY is one day in Windows filetime format.
time = GetLocalTime(); /* get the local time */
month = MakeLowerCase(FormatDate(time,"M")); /* get current month */
if (month == "jan" || month == "feb" || month == "mar"){ /* if q1 */
quarter = "4"; /* use previous quarter */
} /* */
if (month == "apr" || month == "may" || month == "jun"){ /* if q2 */
quarter = "1"; /* use previous quarter */
} /* */
if (month == "jul" || month == "aug" || month == "sep"){ /* if q3 */
quarter = "2"; /* use previous quarter */
} /* */
if (month == "oct" || month == "nov" || month == "dec"){ /* if q4 */
quarter = "3"; /* use previous quarter */
} /* */
year = FormatDate(time,"o"); /* get year */
if (quarter == "4"){ /* if we want Q4's list */
year = FormatString("%d",TextToInteger(year)-1); /* subtract a year */
} /* */
cusip_loc = ReplaceInString(CUSIP_LIST_BASE,"YYYY",year); /* add year to cusip string */
cusip_loc = ReplaceInString(cusip_loc,"QQQQ",quarter); /* add quarter to cusip string */
The first thing our script needs to do is build the location to check for the file. We can use the GetLocalTime SDK function to get the current time. Using the FormatDate function, we can get the current month as a 3 letter abbreviation, and make sure it’s lower case with the MakeLowerCase function. If it’s the first 3 months of the year, we want to use Q4’s list. Otherwise, we just set quarter to the previous quarter in this year. We can use the FormatDate function to get the year as well. After we have the current year, we need to check if we’re getting Q4’s list. If so, we want to subtract a year from our current year. Because year is a string, we need to convert it to an integer value first with the TextToInteger function and then back to a string with the FormatString function. Once we have both the quarter and year, we can use the ReplaceInString function to build the location to test on the SEC’s site.
appdata = GetApplicationDataFolder(); /* get the appdata folder */
product = GetApplicationName(); /* get the application name */
settings = product + " Settings.ini"; /* get name of settings file */
settings = AddPaths(appdata,settings); /* get path to settings */
current_list = GetSetting(settings,"Form 13F Options","PDF CUSIPs");/* get current cusip list */
library_loc = GetSetting(settings,"EDS","Library"); /* get library location */
old_cusip_csv = AddPaths(library_loc,CUSIP_CSV); /* get path to old cusip csv file */
if (current_list == cusip_loc){ /* if we have the most up to date list */
return ERROR_NONE; /* return without error */
} /* */
Next, we need to get our settings file name so we can check if we need to update the CUSIP list or not. Using the GetApplicationDataFolder and GetApplicationName functions, we can build a path to the settings file so we can read settings out of it. We’re also going to need to get the path to the old CUSIP file, which is in the library directory. So we pull the library directory out of the settings file and add it to our defined file name to get the path to the library file. Then, using the GetSetting function, we need to check the setting named “PDF CUSIPs” to see if it matches our generated location on the SEC’s site. If it matches, we already have the most up to date list, so we can just return without error.
last_attempt = GetSetting(settings,"Form 13F Options",LAST_ATTEMPT);/* get last attempt to dl file */
if ((time - last_attempt)<ONE_DAY){ /* if less than one day has passed */
return ERROR_NONE; /* return without trying again */
} /* */
tempfile = AddPaths(GetTempFileFolder(),TEMP_FILENAME); /* get path to temp file */
rc = HTTPGetFile(cusip_loc,tempfile); /* get temp file */
DeleteFile(tempfile); /* remove temp file */
if (rc!=200){ /* if we failed to get the file */
PutSetting(settings,"Form 13F Options",LAST_ATTEMPT,time_s); /* store last attempted dl time */
return ERROR_NONE; /* return without error */
} /* */
PutSetting(settings,"Form 13F Options","PDF CUSIPs",cusip_loc); /* store new cusip list */
DeleteFile(old_cusip_csv); /* delete the old CUSIP CSV file */
LoadPreferences(); /* force preferences to reload */
return ERROR_NONE; /* return without error */
If our file is out of date, we can get the last attempted download with the GetSetting function. If it’s less than one day ago, we want to return without error again, because we already checked today. Otherwise, we can build a path to a temp download file. Then we can use the HTTPGetFile function to try to download the file using our link to the temp file directory. Immediately after download, we can just delete the file, because the actual contents of the file are irrelevant. All we care about is whether or not we could download the file. For that, we just need to check the return code of the HTTPGetFile function. Return code 200 means that everything is OK, and we got the file successfully. Anything else means an error happened and we cannot download the file successfully, so we need to store the last attempt in the settings file before returning.
If we did download the file without a problem, we can again use the PutSetting function to put it into our settings file and then delete the old CUSIP file to force the application to re-download and parse it when the user next presses validate on a CUSIP list. Finally, we can call the LoadPreferences function to force our settings to re-load. This initialization function has GoFiler check every time it’s run to see if it has the most recent CUSIP file from the SEC, and if not, it attempts to download it. This a very appropriate use of the Application Initialization script to keep our application up to date.
 Steven Horowitz has been working for Novaworks for over five years as a technical expert with a focus on EDGAR HTML and XBRL. Since the creation of the Legato language in 2015, Steven has been developing scripts to improve the GoFiler user experience. He is currently working toward a Bachelor of Sciences in Software Engineering at RIT and MCC. |
Additional Resources
Legato Script Developers LinkedIn Group
Primer: An Introduction to Legato
Quicksearch
Categories
Calendar
Feeds
