New Century and Integrity Plus Blog

Boost Your Data

Posted by Jennifer on Sep 12, 2018 9:00:00 AM

It takes a lot of work to gather, organize, and manage asset data. Whether the data is old or new, things can get messy quickly. We’ve got some tips on how to make the data management process more efficient through converting data to a common format and centralized location, data cleanup, and making use of views and queries. 

Extreme Makeover: Data Edition

Convert physical records to a digital format

This can be a daunting task, so take a deep breath and make a game plan. Assign as many resources as possible to this task and limit the data being converted to what is really important. Even though it is tempting, you don’t need everything; prioritize and stay focused on the end goal. During the process, be sure to document all actions and QA/QC the converted data in ArcMap.

Get data into a single location

Don’t spend time searching multiple places and files for that one piece of data. Make it easily accessible for everybody by knowing who the stakeholders of the data are, and then find a common reporting or data sharing location. This lessens time-consuming queries on the integrity department and makes for a more optimized workflow.

Handling large data sets

Staring down hundreds of columns and rows of data and not sure where to start? Turn to proven processes from experience within the company, a vendor, or consultant, then determine what data is the most important. Be careful, don’t make any assumptions! Take the time to research, chat with field personnel, and meet with a SME to determine the core, most useful data to users. Once you’ve sorted this out, test with a small group of data to make sure you’re getting what you wanted and it’s meeting the organization’s needs.

Clean Up, Aisle 5

Get rid of the ‘junk’

Believe it or not, more isn’t always better. Take the time to sort through the madness to remove gaps and overlaps, duplicates, centerline corrections and domains. It also helps to consider what is really necessary when it comes to the level of detail and precision for certain data points. Going through these steps can relieve storage issues, lead to better database performance, and elevate the value of reports and analyses.

Create structure

Good data is organized data. It takes a complete system that includes tools, people, and processes to properly manage data. First things first, identify who will be responsible for the data. This can be one person or a committee of people. Next, establish a process for who governs what, how data is being captured into the system, where the data is coming from, and who validates it. With a clear path and organization of files, maintaining and utilizing data will be a breeze.

Views, and Queries, and Derived Layers, Oh My!    

Sometimes people want an answer, and they want it now. Through the use of pre-determined views, queries and derived layers, you can provide that johnny-on-the-spot data and increase your organization’s efficiency.  This can be essential when it comes to emergencies and incidents to get pipeline specs with a single click. Business Development and Integrity groups can utilize these, as well, to see multiple tables of data at once and perform quick assessments and analysis.


In the end everything comes back to data, so having good, clean, accessible data is paramount. Put in the effort and, trust us, it’ll be worth it!

Want to alleviate the pressure of establishing an optimized data platform? Leave it the experts and contact us for all things data management, from modeling to conversion to onsite staffing

Read More

Topics: data quality, Data Management, data cleanup, data conversion

Back to the Future: Reconciling Pipeline Data with ILI to Improve Data Quality

Posted by Lynette Lyddane & Sarah Spaulding on Feb 28, 2018 9:16:36 AM

In the classic film Back to the Future, Marty McFly accidentally goes back in time, returning to 1955 when his parents were teenagers. To get “back to the future”, his future, he must make sure that his parents meet and fall in love. Of course, there are hijinks along the way that threaten his very existence. With each misstep, he notices his siblings start to disappear from the family photograph he carries in his pocket, as he changes the course of history.

Read More

Topics: ILI, Pipe Data, data quality, Alignment Manager

Subscribe to Email Updates

Recent Posts

Posts by Topic

see all

Follow Us