Revision as of 01:22, 5 November 2015

1 Overview

MetaData is a dedicated data module in bxp software. It allows data from numerous sources to be retrieved, transformed and pushed out to reports and other systems. Each stage of the process allows for custom rules to be applied. The stages are Retrieve, Input, Process and Output.

MetaData is complicated and requires technical expertise. The formal documentation on MetaData can be read here. File:Module - MetaData v2-0.pdf

2 Background

Combining data from numerous Excel spreadsheets to generate one comprehensive report is now just a click away. Our clients have already begun to use MetaData to save hundreds of operational hours in report generation and replacement of laborious, time consuming tasks. Here are some examples:

Combining data from numerous sources such as phone systems, HR systems, quality assurance scores, and countless other sources, to combine the data and calculate a score which could be loaded into a payroll system are now easily managed.

Using the highly modularised approach, each process is broken down into simple individual steps called rules. Rules are combined up into sequences. Sequences are combined up into Programs, which enables very complicated processes and procedures to become a single click of a button, or even an automated task. ISO compliance and process management with full audit trails is now a matter of a simple once off setup and guaranteed compliance.

Have your report sitting waiting for you to collect first thing on Monday, while all of your data retrieval, collation, processing and output formatting is done on a Sunday by bxp software. Make Monday meetings more effective with all your reports ready and waiting to go.

With automated data outputs now you can easily have a web front end to anything from an AS400 to integrating live stream technologies. bxp software makes data processes simpler.

3 Understanding storage

The elephant in the room is that the person using the MetaData module must have an understanding of databases. This module and the following operations require you to have a working understanding of database Structured Query Language (SQL) and also understand what goes into manipulating raw data. This is often done by reporting staff in organisations worldwide and does required a technical skillset that is often considered by numerous organisations as specialised.

Within bxp we use the MySQL database engine. For some of the nuances of MySQL can be found here: http://php.about.com/od/learnmysql/ss/mysql.htm

4 Structure of Processing

• A rule is an instruction to be performed.
• A sequence is a set of rules to be performed on all records or a subset of records.
• A program is the sequential execution of a number of sequences
• A sequence can belong to more than one program.

5 Types of Processing

• Retrieve
  • Gets the raw data into the BE infrastructure by what whatever medium available and chosen.
• Input
  • Field mapping and processing rules for extracting data from Excel, structured data or XML documents
• Process
  • Can be considered a number of stages which may / may not be required.

1. Initial data setup
2. Ensuring data is all on one line
3. Data Tidy up
4. Data conversion
5. Data transfer

• Output
  • A sequence of post processing tasks including communications, report generation and follow up tasks.

6 Development sequence

When building a MetaData program a general build approach can be adopted. This document has been built to follow the general steps in constructing a program. A worked example using a Nortel phone system report is provided to attempt to make easier to understand the stages involved. 

7 Preparation: Create the initial data source storage structure

Before you can do anything with the system you need to have a database to store the data in. If you’re using an existing database structure such as an Inbound, Outbound, Blended campaign or Testing Centre Exam or QA program, then you have a structure to work with already.

The easiest thing to do is to upload an Excel file manually and create the database structure from the upload. This is fine if your data source is an Excel spreadsheet. If you are working with other formats, convert them into a spreadsheet first. This is a suggestion but saves a lot of time and precision issues.

Main Menu > Metadata > Data Management > Create Database from Excel Data

You can view the loaded resulting data next to see how when the data has been cleaned up, it is represented in the system.

Main Menu > Metadata > Data Management > Data - Export