LDC #67: Legato Program Design Best Practices

Friday, January 12. 2018

LDC #67: Legato Program Design Best Practices

In past blog entries I’ve written, the topic is often about simple scripts. It’s covered a couple hundred lines of code, and the script usually handles a single function. This means that every script that’s been covered so far has been relatively simplistic in structure, with no more than 10 or so separate functions. With so little code, a script can still be easily maintainable even if no thought is given to how it’s put together. The design of the application is less important in these cases. However, what if you want to do a larger project? What if your script is going to include multiple UI screens and integrate with multiple systems? This is where you would want to pay some attention to design and apply some software design practices to the script to ensure that you get the best possible result. The blog post this week will not contain an example of the script but will rather discuss how we can apply some software engineering principles to Legato design to avoid common pitfalls.

The Project - SEC Filing Viewer

The SEC requires a public company to post their XBRL on their website. Well, wouldn’t it be great if GoFiler could just do it for you? Let’s assume we want the ability to push (using FTP) a project file’s contents to a website in a way that’s clear and easy to understand. We would also want the ability to view and edit filings we’ve pushed to this website in the past. This could be a very large undertaking because it has a lot of moving parts. Just starting to code without considering the overall design would probably result in a lot of frustration and very little progress being made.

1) What will I need before I get started?

Let’s assume we already have a company website. We need that company website to have a folder that will allow a visitor to access any HTML files in the folder. We will need that company website set up to allow FTP access from the computer on which it will be running so we can actually upload files. Setting this up is a bit beyond the scope of this blog post, but any webmaster should be able to do this.

2) What exactly do I need to do?

That seems like a fact worth knowing. Don’t start programming until your requirements are crystal clear. We need to break it down into chunks so we don’t get confused halfway through. We’ll need a few separate things:

• An HTML Template for a project file is the first thing we need. The project has some data points in it, like CIK, Filer, Series / Classes, SROS, etc. We need to decide what data points we want to display for each filing and design a template that will show these to a user in a clear way. We also need to make sure the template is pretty universal so that it would work for an 8-K filing, a 10-Q filing, and anything else a company might file. We also need a section for links to documents included in the filing. This is mostly a web design activity, but it’s a crucial step in the process.

• We need three menu functions added into GoFiler. One is a function to upload a project, which should run only when a project file is open and it should create a copy of the project file template with the appropriate data filled in. It should also have links for each of the files included in the filing. It would then need to push these files to our FTP server. The second menu function would be an option to view all filings on the FTP server and to delete or edit basic properties about them. The final function would be an “Options” screen for the operation, which should have fields for the FTP host, the password, and any other options you may want to add in. Each of these functions should be further broken down into its constituent requirements, so it’s very clear as well.

• We should probably have an XBRL previewer as well. Uploading raw XBRL XML files is all that the SEC requires, but it would be best to have an XBRL rendering for visitors to view. GoFiler has an XBRL Previewer built in Legato, which installs the XBRL Renderer from the SEC and compiles the results into a single file. Integrating this into the uploader menu function would probably be the easiest way to go, but we could use any XBRL rendering engine we want so long as it produces HTML files the uploader can post to the website.

Avoiding the Anti Patterns

When working on a system like this, we have a couple of different components to keep straight. In software development, design patterns are common approaches to solving problems. For example, you can apply the singleton design pattern to ensure only a single instance of a class or object exists in your application, and it has global access. This might be useful if we want to make a global handle to our edit object, and ensure it’s the only edit object handle we have. On the flipside, there are anti patterns, common pitfalls that are to be avoided. We want to make sure we avoid common software design anti-patterns, or else we could potentially create some very confusing and hard to understand code.

The easiest anti-pattern to overlook, but also easiest to prevent, is the “blob” anti-pattern. In object-oriented programming, a blob is what happens when everything you write is in a single class. Legato isn’t really object oriented, but we can apply a similar definition to having all of your functions in a single file. It makes the code a lot harder to read through and maintain if you have a single file that is several thousand lines long. Instead, breaking out the code into smaller sub files makes it much easier to understand and modify in the future. These smaller files can all be included in the main file, so everything works together. With a project like this, we should break the code up into the following files to make it easier to understand:

• FilingUploader.ms – The main file that includes the others and has the defined values in it. This would also define the function for uploading a file to the web server through FTP with whatever the current settings are.

• UploaderOptions.ls – The file that defines the behavior of the edit options window. It is included by FilingUploader.ms. It should also contain a setup_options function, which the main setup function in FilingUploader.ms calls.

• UploaderOptions.rc – The resource file that defines the dialog for UploaderOptions.ls.

• ManageFilings.ls – The file that defines the behavior of the dialog that displays the filings and allows the user to choose which to edit on the website.

• MangeFilings.rc – The resource that defines the dialog that displays the filings on the website.

• EditFiling.ls – The file that defines the behavior of the edit filing dialog, which allows the user to edit information like the date of the filing, the title, and any other defined properties.

• EditFiling.rc – The resource that defines the dialog for editing the properties.

• ManagerUtilities.ls – A general purpose file for any subroutines that will be used in multiple files. Some example functions would be date converters, debug printouts, debug consoles, file name formatters, and validators. These are functions that may apply to multiple files and aren’t specific to any one subsystem.

This breaks our script into eight separate files. Doing this isn’t required, but it makes it a lot easier to read through the code. This way you know where to look for a specific function to make changes, and it’s more obvious to what part of the script each function corresponds.

Another common anti-pattern to watch out for while developing an application is “Hard Coding”, or putting variables in as hard-coded values. For example, you could get away without using the UploaderOptions files and not have any options in your uploader. The options could instead be just hard-coded into the script file itself, so that you can’t change it. This may meet your initial requirements and get you up and running faster, but it’s really not a good long-term solution. What if someone else wants to change a setting in your script? Expecting someone to go into the source code and edit the variables is not a realistic solution. Without a dialog he or she can use to edit parameters, the script is not going to be very adaptable to future changes.

Documentation: Eating Your Vegetables

Let’s assume you spend a month or two and successfully build your file uploader. Hooray! Good job! Now, two years later, you want to change the way the uploader works so it can use a new format of project file. That’s going to require a few changes to the FilingUploader.ms file. What functions do you change? What do the functions you have in there do? What does this line of code do? It looks like it was written by an insane person!

This is the kind of reaction you can have to software you wrote two years ago if there’s no notes or documentation on how it works. Therefore, there really needs to be some kind of documentation for any project file. While at times a tedious process, documenting software is a very good idea. Either the process gets done at the start, or someone pays for its absence later. For documentation, you generally want two things at the minimum: commenting in the file on the code itself and a design document that says what each part of the script does. The design document should have an overview of every function in the script, along with what the function expects for inputs, what it expects for outputs, error conditions, and a behavior description. This can be really simple or very complex depending on the function. For commenting, you can either comment every few lines, or comment every single line. Here is an example of commenting on every line, and here is one with commenting every few lines. Which you do is a matter of personal preference, but I’ve gotten into the habit of commenting everything. It makes it easier to understand when someone comes back in and modify things.

Writing out the general overview of the components, including their behaviors and parameters, should be done before you even write your first line of code. It’s a lot easier to see potential flaws in the system design when mapping it out this way. What may be so clear in your head while imagining the system will become murky as you start to build it, so getting it all out on paper at the outset of the project is the best way to go.

Final Thoughts

Designing a well thought out Legato script is no different from designing any software application. Attention must be paid to code readability, maintainability, and ease of modification. It’s hard to predict the future, but as long as you have a good design it can make it as easy as possible to reuse parts of it in future implementations. Now, obviously not all scripts are going to require so much forethought during development. All of our blog scripts so far have been relatively small and didn’t need such an in-depth design period. Anything up to a couple hundred lines of code is pretty simple in terms of software design. Once you start getting into the thousands of lines, unless you have a firm design in place the end product is likely to get sloppy. Applying these basic software development ideas to larger Legato projects can improve development speed and ensure a higher chance of a successful software project.

Steven Horowitz has been working for Novaworks for over five years as a technical expert with a focus on EDGAR HTML and XBRL. Since the creation of the Legato language in 2015, Steven has been developing scripts to improve the GoFiler user experience. He is currently working toward a Bachelor of Sciences in Software Engineering at RIT and MCC.