The Maryland Volunteer Lawyers Service came to us with an existing scraping system that is designed to help MVLS provide additional information to clients who may not know about legal issues and remedies available to them, such as criminal record expungements and delinquent water bills. The system scrapes various public sites to identify these issues and is connected with the MVLS case management system. Over the years, these scrapers became more fragile and it was unclear how to resolve errors. MVLS came to us to rebuild this product.
The updated system scrapes court cases state-wide, parses the case html, and saves the individual data points into a database. The system is also set to re-scrape/update cases. T&P not only built the scraper and seven different parsers, but also a user interface that allows non-technical MVLS staff to administer the scraping regime (i.e. controlling what gets scraped at what frequency), and review error logs from those scrapers.
Finally, the CLUE server is integrated with MVLS’s case management system. As a result, the system searches a client’s name and date of birth against a number of databases to find additional information in order to provide additional services to the MVLS client.
“MVLS had a great experience working with Theory and Principle updating our CLUE scrapers and database. Now we are able to better serve our clients, by identifying legal issues they didn’t know they had and helping them to resolve those issues.”
The process began with a thorough review of the existing scrapers and the issues the client is facing with them. From there, we reviewed all sources of data that will be required and identified technical requirements. At the same time, we interviewed users of the system to determine what was needed for the user interface to manage the scrapers.
Once the design was completed and the technical requirements were well-defined, we set out to build the UI, the scraper, and all of the related parsers. Once those were settled, we then worked to integrate the system with Legal Server, MVLS’ case management system.
Following the success of the project, we are now working with the Maryland Governor’s Office of Crime Control & Prevention to organize and index their data to allow for more efficient searching, which will result in an estimated 60% time savings.
