MetaData - Overview

From All n One's bxp software Wixi

Jump to: navigation, search

1 Overview

MetaData is a dedicated data module in bxp software. It allows data from numerous sources to be retrieved, transformed and pushed out to reports and other systems. Each stage of the process allows for custom rules to be applied. The stages are Retrieve, Input, Process and Output.


MetaData is complicated and requires technical expertise. The formal documentation on MetaData can be read here. File:Module - MetaData v2-1.pdf


2 Background

Combining data from numerous Excel spreadsheets to generate one comprehensive report is now just a click away. Our clients have already begun to use MetaData to save hundreds of operational hours in report generation and replacement of laborious, time consuming tasks. Here are some examples:


Metadata overview.png


Combining data from numerous sources such as phone systems, HR systems, quality assurance scores, and countless other sources, to combine the data and calculate a score which could be loaded into a payroll system are now easily managed.


Using the highly modularised approach, each process is broken down into simple individual steps called rules. Rules are combined up into sequences. Sequences are combined up into Programs, which enables very complicated processes and procedures to become a single click of a button, or even an automated task. ISO compliance and process management with full audit trails is now a matter of a simple once off setup and guaranteed compliance.


Have your report sitting waiting for you to collect first thing on Monday, while all of your data retrieval, collation, processing and output formatting is done on a Sunday by bxp software. Make Monday meetings more effective with all your reports ready and waiting to go.


With automated data outputs now you can easily have a web front end to anything from an AS400 to integrating live stream technologies. bxp software makes data processes simpler.


3 Understanding storage

The elephant in the room is that the person using the MetaData module must have an understanding of databases. This module and the following operations require you to have a working understanding of database Structured Query Language (SQL) and also understand what goes into manipulating raw data. This is often done by reporting staff in organisations worldwide and does required a technical skillset that is often considered by numerous organisations as specialised.


Within bxp we use the Oracle MySQL database engine. For some of the nuances of Oracle MySQL can be found here: http://php.about.com/od/learnmysql/ss/mysql.htm


4 Structure of Processing

  • A rule is an instruction to be performed.
  • A sequence is a set of rules to be performed on all records or a subset of records.
  • A program is the sequential execution of a number of sequences
  • A sequence can belong to more than one program.


5 Types of Processing

  • Retrieve
    • Gets the raw data into the bxp infrastructure by what whatever medium available and chosen.
  • Input
    • Field mapping and processing rules for extracting data from Excel, structured data or XML documents
  • Process
    • Can be considered a number of stages which may / may not be required.
  1. Initial data setup
  2. Ensuring data is all on one line
  3. Data Tidy up
  4. Data conversion
  5. Data transfer
  • Output
    • A sequence of post processing tasks including communications, report generation and follow up tasks.