Data Profiling Overview

From All n One's bxp software Wixi

Jump to: navigation, search

1 Overview

Data Profiling is a module that provides all the primary reporting to forms built in bxp.

This article provides a hub to be able to go explore the different types of reports.

2 Background and Keywords

Process. Any activity or set of activities that uses resources to transform inputs into outputs can be considered a process.

Procedure. A procedure outlines how to perform a process

Work Instructions. A work instruction describes how to perform a task, which is a more detailed portion of the procedure such as Completing a PO or Ordering supplies.

There are a number of services which a business can offer. The primary definition of the services stems from which party is doing the contact.

If a customer is contacting the centre, it is called inbound activity. If the agent is contacting the customer it is called outbound activity.

One paper based form constitutes interaction with one customer. (Imagine a paper based passport application form). In a computer system, that one form is called a record. If a customer calls in for a second time, you might have one record, but there would be two entries for the two contacts. Reporting on records and contacts is very important to a business. Every record will have a contact history, which will list all the contacts for that record.

Data in bxp is stored in a collection of database tables which can be referred to as a campaign or database but is called a Form by All n One. Every form has its own unique id known as a Form Id or Campaign Id.

Records and Contacts are known as CDAs and CCLs respectively

A form may have many fields of data. Each field can have its own user entered name. This process of naming fields is called field mapping.

The form structure includes many auto generated and managed system fields.

There are three ways of creating the contents of a CDA record

  • Data Entry
  • Importing data from a file
  • Transfer of data from an external source, such as a website, mobile app or another bxp Form.

Outcomes are used at the end of a script to record contact which logs the CCL against the CDA.

Outcomes have a display category (complete or incomplete) and a number of other important fields including Report Grouping 1 and Report Grouping 2.

bxp has an outbound contact management system which allows for distribution of records.

3 Data Profiling

3.1 Data Profiling Options

Data Profiling is the reporting module for all forms in bxp.

Within Data Profiling there are two different primary ways that the reports are generated: Instant or Customised.

Instant reports are generic with fewer configuration options which allows reports to be generated quickly and with little or no configuration.

When there are a lot of configuration options, you can save the settings and filters in a custom group. These groups are then applied to the instant reports and are called customised reports. These groups save a lot of time by remembering the settings and the group can then be reapplied to any of the report types.

The report types are:

  • Data
  • Statistics
  • Scores
  • Visualisations
  • Reminders and Case Management
  • Export

The Instant or Customised groups created can be applied to a number of different styles of reports.

  • Data reports will return data from the CDA tables or CCL tables where appropriate
  • Statistics reports will return counts, averages and totals
  • Scores reports where quality assurance or testing campaigns are being used, the scoring results will be displayed as reports
  • Visualisations reports generate pie charts and bar charts on the data where closed questions are used

Only available as Instant reports:

  • Reminders and Case Management are operational management reports
  • Export provides the ability to export large volumes of data quickly

Dashboards are grouped collections of reports that address operational reporting needs and they reduce the time taken to navigate to different reports.

3.2 Unique Data Records

Using the demonstration data from CC-1-3, 2.3.2 Form Structure we have a sample CDA table containing

Id Firstname Surname Home Phone Work Phone Mobile Email
1 Nick Wheeler 01 1234567 01 4294000 353871231231
2 Chris Thomson 01 2345678 01 4294000 353871231232
3 Philip Lacey 01 3456789 01 4294000 353861231230

And we have a sample CCL table containing (3 contacts with CDA Id = 1, i.e. Nick Wheeler)

Id CDA_Id Start End Agent Outcome Comment
1 1 2014-01-01 13:00:00 2014-01-01 13:05:00 Agent 1 Interested Wants more info but later
2 1 2014-01-01 14:30:00 2014-01-01 14:36:00 Agent 2 Call back Rang back and gave info
3 1 2014-01-02 10:05:00 2014-01-02 10:12:23 Agent 1 Sale Called and sold service

When running a report for the week of 2014-01-01 through 2014-01-07, do we want the Nick Wheeler account to show up as 1 or 3 records?

If you want a list of all contacts, you would want to see 3 lines of data but if you want a list of all the customers you interacted with that week, you would only want to see Nick’s record once.

Unique data records, or unique records, is the terminology for reducing the output list to only showing a unique CDA once in a report despite having multiple matching CCLs.

The CDA details shown will always be the most recent information.

3.3 Period Selection

Choosing dates and times for reports is a key element of the reports and bxpsupplies a handy assortment of date-time keywords to save on date and time selection.

Clicking the key words of the date time selector ensures that the times of reports are selected correctly. Selectable dates default to 00:00:00 because most reports are set to a day or week period.

It is easy to change the dates and times by clicking the date / time selector icon to the right of the appropriate box. NOTE: The date time can be cleared with the red X to the right of that.

Dataprofiling descriptions 001.png

Where appropriate the reporting time can default to the current date and time. This requires the user generating the report to be vigilant about the time period requested.

NOTE: bxp will always include the start date and time in the report but exclude the end date and time. For example, if 2014-01-07 00:00:00 had been chosen as the end time, 23:59:59 would be included, but 2014-01-07 00:00:00 would not.

3.4 Volume of Data and Integrated Warehousing

When a report is run, a lock is placed on the CDA and CCL. This allows reports and look ups to be performed on the data, but stops data being added to the tables.

For this reason, when big reports are run using Report by Data by Outcome, the tables can become locked which tends to be counter productive during an operational day.

When a campaign exceeds 20,000 records, a feature called Integrated Warehousing (IW) is applied to the campaign CDA and CCL tables. The structure becomes: Dataprofiling descriptions 002.png

Any report generated from the Data Profiling module will check for the existence of the IW versions. If IW exists the report will be performed on the _Report versions of the table, if not it will be performed on the CDA and CCL tables as normal.

As part of using reports in Integrated Warehousing, there is an automated update feature which copies the delta of the records from CDA and CLL to the CDA_Report and CCL_Report tables. This creates a lock to allow the copying to complete. This IW Lock is used to keep records from duplicate copies occurring.

It can also occur that two users try to generate large reports at the same time. The IW lock also locks up the _Report tables until the first user report is finished.

3.5 Closed Vs. Open Questions

A closed question means data entered is from a preselected list such a drop down list or set of radio buttons. These closed question types are easily counted and reports and graphics are readily applied to them.

Open questions allow the user to enter any data they like into the box on screen but they will need to be individually graded by a tutor.

3.6 Reporting Perspectives

Reports in bxp are usually titled from the perspective of the reports that will be generated

  • My reports will limit the returned data to that for the current user.
  • My Department reports will limit returned data to the department the current user is a member of.
  • My Team reports will include all teams that the user is a member of.
  • Customised means that only those records matching specified parameters are returned.
  • All reports will include all information, but with limitations such as time used as a safeguard against the generation of enormous reports.

NOTE: The Data Profiling module reports will always be from the All perspective.

4 Discussions

Time is a huge factor in reports and there are some important differences to note between reports.

Data Profiling - Reporting and the challenge of timed snapshots

5 Jump offs