The data definition (Query) Wizard lets you Create and Modify data definitions used to populate inmydata from the inmydata Data Publisher Client Application.

 

To Create A New Definition click on the 'Add Definition' from the menu or the context menu for a task. To 'Edit Definition 'select Edit Data Definition from the Data Menu of the Task contect menu.


The data definition Wizard is then started and the Data Definition - Introduction screen is displayed.


Data Definition - Introduction


 

The Data Definiton Wizard is started when you choose to 'Add Definition' or 'Edit Definition' from the inmydata Data Publisher Client Application.

 

Click on the check box labelled 'Skip this screen in the future' if you do not wish to see this screen again.

 

Click on the 'Next >' button to continue with creating or modifying your Data Definition.

 

clip0074

 

You can click on the 'Cancel' button (at any time) to leave the Data Definiton Wizard without saving your changes.

 


Data Definition - New or Copy Existing

 

To preserve the integrity of your data definition and publishing tasks, the inmydata Publisher bind the data definitions to individual tasks and prevent them from being accessed as a separate entities. You can either create a new Data Definition to base your publishin task on, or you can take a copy of an existing shareed Data Definition.

 

Create a new query is the default setting.  If you select this option and click on 'Next>' you will proceed through the data Definition Wizard creating a new data definition which will then be bound to your task and used as its data source.

 

Copy an existing query If you click on this option you can then click on the browse button.  A standard Windows 'Open' dialogue box will appear and you can select the data definition/query (.qpa) you would like to take a copy of.  Click on 'Next>' to then proceed through the Query Wizard and Modify the copy.  The copy is then bound to the task leaving the original query intact for others to use as per normal.

clip0118

If at any stage you wish to step '<Back' or 'Cancel' what you are doing, click on the appropriate button.



Data Definition - Name and Description


The Data Definition Wizard - Name and Description form lets you give your Definition a name and a suitable description.  You must enter a name, but the description is optional.

 

clip5844

 

Click on the 'Next >' button to move on.  If you are creating a new Definition you will move on to the Query Wizard - Select Type form, if you are modifying an existing Definition you will move on to the Data Definition Wizard's page for your Definiton Type (see Types for the appropriate type) form.

 

Click on '< Back' to return to the Data Definition Wizard - Introduction form.

 

You can click on Cancel (at any time) to leave the Query Wizard without saving your changes.


Data Definition - Select Type


The Query Wizard - Type form lets you select the Type of Query that you want to create

 

clip5847

 

 

Standard Query

A standard query will return a row of data for every database record retrieved by the query. Use this type of query when when you want to create a dashboard where user can drill down to the lowest level of detail (for instance right down to the order line). The disadvantage of these queries is they return significant amounts of data, which may impact performance if the quantity of data is very high.

Click on the 'Next >' button to move on to the Query Wizard - Select System screen

Summary Query

A summary query will summarise the data on the server before it is returned to the dashboard. Use this type of query when you want to create a responsive dashboard based on large data sets. The disadvantage of these queries is the user will only be able to drill down to the level you have summarised at.

Click on the 'Next >' button to move on to the Query Wizard - Select System screen.

Trending Query

A summary query will summarise the data on the server before it is returned to the dashboard. Use this type of query when you want to create a responsive dashboard based on large data sets. The disadvantage of these queries is the user will only be able to drill down to the level you have summarised at.

Click on the 'Next >' button to move on to the Query Wizard - Select System screen

Freeform SQL Query

A freeform SQL query allows you to manually enter an SQL query to retrieve data

Click on the 'Next >' button to move on to the Query Wizard - Select System screen

Import data from a file

This query type allows you to import data from common file types such as Excel and csv files

Click on the 'Next >' button to move on to the Data Definition - Import from a File screen

Blending Query

A blending query allows you to blend data from two or more seperate sources into a single data table

Click on the 'Next >' button to move on to the Data Definition Wizard - Select Blend Sources screen

Google Analytics Query

A Google Analytics query allows you to create queries that extract data from the web analytics service offered by Google that tracks and reports website traffic.

 

Open API Query

An OpenAPI query allows you to create queries that call a method in a bespoke assembly to generate data. They allow developers to build their own data sources.  For more details on using this type of query, see How do I create an open API query?


Click on the 'Next >' button to move on to the  Data Definition Wizard - Enter Class screen 


 

 

If you are creating a query on an ODBC based system then the Query is always a Standard Query but you do have the option of not selecting a Subject. If you do this then you will be taken to a step in the Query Wizard where you can enter an SQL select statement that will be used to retrieve data.

  

Click on '< Back' to return to the Data Defintion Wizard - Introduction form.

 

You can click on Cancel (at any time) to leave the Query Wizard without saving your changes.



Data Definition - Select System


Each data definition runs against a System, and so you need to define which System that is.

 

To select a System click on the System Combo box button to reveal a drop down list of the Systems that are available to you, then click on the name of the appropriate System.

 

If the System has a description, it will be displayed in the list view underneath the combo box.

 

This form only appears when you first Create a Data Definition.  You can not change the System that an existing data definition runs against.

 

clip5845

 

If the system you want is not displayed then click on "Click here to add a System' to open the System Wizard - Introduction

 

If the system selected has more than one connection set up for it then the 'Always use selected connection for this query' check box will appear below the System Description.

 

If the you check this check box then a drop down list of the connections that are available to you for the selected system will appear. If you select a connection here then the selected connection will ALWAYS be used regardless of what the default or current connection is.

 

Click on the 'Next >' button to move on to the Data Definition Wizard - Select Subject form.

 

Click on '< Back' to return to the Data Definition Wizard - Name and Description form, or click on Cancel at any time to leave the Query Wizard without saving your changes.



Data Definition - Select Subject


 

Each Query runs against specific Subjects. It is not possible to change an existing Query so it will run against different Subjects.

 

To select a Subject, Click on the Subject Combo box button to reveal a list of Subjects belonging to the System you chose in the previous screen.  In the drop down list, click on the name of the Subject you wish to use.

 

You can also type part of the name of the Subject you are looking for into the Subject Combo box itself. This will filter the Subject in the Subject Combo box down to just those which match the string you have typed.

 

Alternatively, if your system contains a lot of subjects, you can use the Subject Explorer to select a subject by pressing the subject explorer button.

 

clip5846

 

If you have selected a System that is based on SQL then you can press  'Next >'  to create a Freeform SQL Query.

 

Once you have chosen a Subject, click on 'Next >' to move on to the next step, the Data Definition Wizard - Define Filter Conditions form.

 

Click on '< Back' to return to the Data Definition Wizard - Select System form, or click on 'Cancel' to leave the Query Wizard without saving your changes.

  


 

Data Definition - Define Filter Conditions


The Query Wizard allows you to define Filter Conditions so you can narrow down the range of data that your Query will extract from the application database.

 

If you do not wish to build any Filter Conditions into your Query, simply click on 'Next >' to move to the Data Definition Wizard - Select Columns  form.

 

clip5848

 

To Add Filter conditions click on 'Add' and the Query Conditions form will open up.

If you wish to Edit or Delete any of your Query Conditions, select the condition by clicking on it in the list view.  Then click on either the Edit button or the Delete button.  If you click on the Edit button, this will open the Query Condition form.

 

clip5849

Where you have multiple conditions, especially combinations of  'AND' and 'OR' conditions, it is possible to add parenthesis to group certain conditions together, by using the 'Group' button.

 

clip5850

 

Drag the mouse over the required conditions to select them, and then click on the 'Group' button.  Parenthesis (Brackets) will then be placed around the selected conditions.  To Remove the parenthesis, select the grouped fields and click on the 'Ungroup' button.

clip5851

 

Once you have defined your Query Filter Conditions click on the 'Next >' button to move on to the Query Wizard - Select Columns form.

 

Data Definition - Select Columns


Depending on the type of Data Definition (Query) you have selected there are 3 


  1. Standard Data Definition 
  2. Summary Data Definition 
  3. Trending Data Definition 



Data Definition - Select Columns (Standard)


You have already selected the Subject that you wish to base your Query on and that you wish the Query to be a standard one, however the Subject may contain data Fields that you do not wish to be included in your Query.  The Query Wizard - Select Field form lets you choose which Fields will be included in your Query, and in what order they will be included.

clip0082

SELECTING FIELDS

 

The screen displays a List of the fields that are 'Available' for selection, and a list of Fields that you have already selected.  To select a Field, first highlight it by Clicking on it in the Available list, and then click on the '>' button to move it over to the Selected list. You can also do multiple selections by holding the mouse button down and dragging the mouse over the required fields.

 

You can also use the box immediately above the Field lists to enter part of the Field name that you are wanting to Add/Remove. As you type in this box you will notice that the list of Fields will be filtered based on the what you have typed. This is a useful way of searching for the field you want if there is a large number of Fields in the Subject you are using for this Query.

DESELECTING FIELDS

 

You can deselect any of the Selected Fields, or change their listing or order in the Selected list.  While you can deselect multiple Fields you can only change the order of one field at a time.

 

First highlight the Field/s in the Selected list in the same way that you can highlight fields in the Available list.  Then click on '<',  If you click on '<<' all Field/s will be de-selected and moved back into the Available list.

 

DATA PREVIEW

 

You can also select any fields displayed in either the Available Fields or Selected Fields list and select the Data Preview Icon. This gives options for adding or removing fields from each of the list but it also gives you the option to preview some of the data in the Data Preview screen.

 

  

Once you have finished making your selections click on the 'Next >' button to move on to the Data Definition Wizard - Sort Criteria form.

 


Data Definition - Select Columns (Summary)


You have already selected the Subject that you wish to base your Query on and that you wish the Query to be a Summary one, however the Subject may contain data Fields that you do not wish to be included in your Query.  The Query Wizard - Select Field form lets you choose which Fields will be included in your Query, and which which of those fields should be used for grouping the data and which of those fields should be used to calculate summaries.

 

clip561

 

 

SELECTING FIELDS

 

The screen displays a List of the fields that are 'Available' for selection, and a list of Fields that you have already selected.  The Fields that have been already selected are divided by into two lists. The Group by list and the Summarize list.

 

To select a Field to group by, first highlight it by Clicking on it in the Available list, and then click on the top '>>' button to move it over to the Group by list.

 

To select a Field to summarize by, first highlight it by Clicking on it in the Available list, and then click on the bottom '>>' button to move it over to the Summarize list.

 

You can select the type of summary to be used for the highlighted Summary Field by using the Summary Type drop down list.

 

Depending on the data type of the highlight Summary Field different items will be available in the Summary Type drop down. For example numeric fields will have Sum, Maximum, Minimum, Average, Count, Distinct Count, First and Last but non numeric fields will only have Count, Distinct Count, First and Last.

 

You can also use the box immediately above the list of Available Fields to enter part of the Field name that you are wanting to Add. As you type in this box you will notice that the list of Available Fields will be filtered based on the what you have typed. This is a useful way of searching for the field you want if there is a large number of Fields in the Subject you are using for this Query.

 

REMOVING FIELDS

 

To remove a Group Field, first highlight it by Clicking on it in the Group by list, and then click on the top '<<' button.

 

To remove a Summary Field, first highlight it by Clicking on it in the Summarize list, and then click on the bottom '<<' button.

 

SUMMARY GROUP OPTIONS

 

You can set Summary Group Options for the highlighted field in the Group by list by clicking on the Group Options button. More details on Summary Group Options can be found in the Summary Group Options topic here.

 

DATA PREVIEW

 

You can also right click on any of the fields displayed in either the Available Fields, Group by or Summarize lists and a pop up menu will appear. This gives options for adding or removing fields from each of the list but it also gives you the option to preview some of the data in the Data Preview screen.

 

Once you have finished making your selections click on the 'Next >' button to move on to the Data Definition Wizard - Sort Criteria form.

  


Data Definition - Select Columns (Trending)


You have already selected the Subject that you wish to base your Query on and that you wish the Query to be a Trending one, the Subject may contain data Columns that you do not wish to be included in your Query.  The Query Wizard - Select Columns form lets you choose which Columns will be included in your Query, and which which of those columns should be used for grouping the data and which of those fields should be used to calculate the trending.

 

 

clip5858

 

  

clip567

 

The options available for creating a trend on are detailed in table below:

 

Option

Description

Top N of

Setting this option limits the data to just the Top N records for this trend based on the type of trend selected, the Summary Field value selected in the Trend Based On drop down list and the value entered in the Where N is box.

 

So, for example, if you enter 5 for the value in the Where N is box, the group field is Country and the date field is Year of Date Ordered, the type of trend selected is based on Growth of and the Trend Based On field selected is Sum of Extended Price then you will get results from your query, for that group, of the top 5 counties whose Sum of the Extended Price has grown most from the first year to the last year over the period of order data being queried.

Bottom N of

Setting this option limits the data to just the Bottom N records for this trend based on the type of trend selected, the Summary Field value selected in the Trend Based On drop down list and the value entered in the Where N is box.

 

So, for example, if you enter 5 for the value in the Where N is box, the group field is Country and the date field is Year of Date Ordered, the type of trend selected is based on Growth of and the Trend Based On field selected is Sum of Extended Price then you will get results from your query, for that group, of the bottom 5 counties whose Sum of the Extended Price has grown the least from the first year to the last year over the period of order data being queried.

Top Percentage of

Setting this option limits the data to just the Top N % of records for this trend based on the type of trend selected, the Summary Field value selected in the Trend Based On drop down list and the value entered in the Where N is box.

 

So, for example, if you enter 10 for the value in the Where N is box, the group field is Country and the date field is Year of Date Ordered, the type of trend selected is based on Growth of and the Trend Based On field selected is Sum of Extended Price then you will get results from your query, for that group, of the top 10% of counties whose Sum of the Extended Price has grown most from the first year to the last year over the period of order data being queried.

Bottom Percentage of

Setting this option limits the data to just the Bottom N % of records for this trend based on the type of trend selected, the Summary Field value selected in the Trend Based On drop down list and the value entered in the Where N is box.

 

So, for example, if you enter 10 for the value in the Where N is box, the group field is Country and the date field is Year of Date Ordered, the type of trend selected is based on Growth of and the Trend Based On field selected is Sum of Extended Price then you will get results from your query, for that group, of the bottom 10% of counties whose Sum of the Extended Price has grown the least from the first year to the last year over the period of order data being queried.

 

Next you need to select what type of trend you want to look for.

 

clip568

 

The two options for trend based on are detailed in the table below:

 

Option

Description

growth of

This option allows you to search for the trend of growth in the data and it can be used for both positive growth (Top N and Top Percentage) and negative growth (Bottom N and Bottom Percentage). To come to a growth figure for each group it calculates the first and last values based on the time series provided by the date field used and considers the difference between these.

variability of

This option allows you to search for variability in the data. To come to a variability figure for each group it calculates the difference between each value in the time series provided by the date field used and sums these differences to provide a variability index. This value is not returned in the data, it is just used to identify the most or least variable groups.

 

The trend based on can be based on the following options, these options will vary dependent on the column's data types,

 

Option

Option

Option

Sum of

Average of

Maximum of

Minimum of

Count of

Distinct Count of

First of

Last of


 

which can be from any of the available listed columns.

 

 

You can now set the period of time query is to be over.

 

clip5860

 

Firstly select one of the available date/datetime columns, select select the grouping for the period of time

 

Option

Option

Option

Second

Minute

Hour

Day

Week

Month

Year/Month

Quarter

Year

 

Further Grouping options can be set using the Group By Screen

 
The other options are detailed in the table below.

 

Option

Description

Trend Based on

This option can be set by selecting a field from the top right drop down list. In this list will be each field that has been added as a Summary Field back in the Query Wizard. The Field selected here will be the one used in calculations when identifying the trend required for this group.

Where N is

This box is where you enter the values that are used with the Top N, Bottom N, Top Percentage, or Bottom Percentage to identify the number of records you are interested in for this group.

Include Ties

The Include Ties option allows you to decide if you want to include records whose values are the same in your results. So for example if you set a Top N to give you just the Top 5 Countries whose order revenue has grown over the last 5 years and the Country in position 5 and the Country in position 6 have the same growth value then setting the Include Ties will mean you see both the Country in position 5 and the country in position 6 in your results. If you set Include Ties to be off then you would only see the Country in position 5 in your results.

 

 Once you have set your Trend, click on the 'Next >' button to go to the Sort Criteria Screen  .

 



Data Definition - Sort Criteria


In the Data Definition Wizard - Select Columns screen you chose the Columns for which data will be retrieved and displayed by your Data Definition.  The Data Definition Wizard - Sort Criteria screen lets you choose the fields by which you wish to sort the data your Data Definition retrieves.  The fields you choose to sort the data by, do not have to be field you have selected to display.

 

As with the Select Columns screen, Filtering of the lists and viewing of data are available.

clip5852

As with the Query Wizard - Select Fields screen, you select Fields in the Available list, and then you can change the order of, or deselect the Fields in the Selected list.

 

The order of your Selected fields is very important.  The records will first be sorted into groups by the first field in the list.  Each group of records will then be sorted by the next field selected, and so on.

 

You can change the sorting order on each of the Fields you have selected.  Click on the Field in the Selected List, and then click on the 'Ascending' check box.  If the check box is checked then the data will assort in ascending order (eg 1 - 10). if the check box is unchecked the data will sort in descending or (eg 10 - 1).

Once you have defined your Sort Criteria click 'Next >' to move to the next screen, Query Wizard - Add Multiple Subjects to Query.

 


Data Definition - Add Multiple Subjects


The Data Definition Wizard - Add Multiple Subjects to Data Defintion screen lets build more complicated Data Definitions by combining multiple Subjects within a Data Definition. 

 

The base Subject of the Data Definition you have been building is displayed in the top left corner of the treeview.  If you do not wish to create a Multiple Subject Data Definition simply click on 'Next >' to go to the Data Definition Wizard - Publish Settings screen.

 

clip5853

 

To Add a child Subject to an existing Subject, click on the existing subject to select it and then click on the 'Add' button.  The Query Wizard will go through the necessary screens so that you can Select a Subject, Define Filter Conditions, Select Fields and define Sort Criteria for the additional Subject's Query as you did for the initial part of the Query.

 

The only difference in the process is in the defining of the Filter Conditions where the Query Condition form requires the parent and the child Subjects to be related to each other.  For details see Data Definition - Multiple Subject Conditions.  Once you have added the child subjects they will be displayed in the treeview.

You can not Delete or Edit the Base subject's query from the Multiple Subjects Form

 

clip5854

To Delete or Edit the additional Subjects, or Add children to them, click on the Subject in the treeview and then click on the appropriate button.

 

clip5855

The Split Query Results option allows you to return each subject as a separate recordset, rather than the default which is to allow inmydata to combine the results into a single recordset. 


The Inner Join option allows you to control the join between the highlighted subject and it's parent. If you select the Inner Join check box then the parent record and any of it's parents will not be include in the results if there are no records returned by the child subject. If you NOT select the Inner Join check box then the parent record and any of it's parents will  be included in the results even if there are no records returned by the child subject.

 

clip400

Once you have finished building your Multiple Subject Query, click on the 'Next >' button to go to the Data Definition Wizard - Publish Settings screen.



Data Definition - Multiple Subject Conditions



When building a Multiple Subject Query at least one of the Query Conditions should relate a column in the Child Subject to a column in the Parent Subject, Report or Dashboard.  As well as Conditions which relate the two Subjects you can also build Conditions which compare a Column in the Child Subject with a fixed Expression or Value.

 

clip5856

TWO SUBJECT CONDITION

 

Using the first combo box select a Column in the Child Subject.

 

Then select a Qualifier.

 

As long as there are compatible Columns available in the Parent Table, select the appropriate Column from the last combo box.

 

Then Click on 'OK'.

If there are no Compatible Columns in the Parent Subject to match the Column chosen from the Child Subject, the Parent Subject Column combo box will be disabled and you will have to create an Expression Condition.  Even if Parent Subject Columns are available you can still opt to use an Expression Condition.

 

clip5857

EXPRESSION CONDITION

 

Select the desired Column in the Child Subject Column combo box.

 

Then select the appropriate Qualifier.

 

If you are not forced to create an Expression Condition, you can still do so by clicking on the Expression check box.

 

Type your value or expression in the panel provided,or generate using the column and function options, that will be evaluated when the query is run.

You can enter any valid Progress expression. Then click on 'OK'.

 

If you need to create an Expression Condition which is a function of, or contains one of the columns in the Parent subject, select the column from the available columns list.


Data Definition - SQL Select


In the previous step the Data Definition - Select Subject you chose not to select a Subject from the drop down list and therefore decided to base the Query on an SQL select statement.

 

clip5859

 

The SQL select statement that will be used to retrieve data for this query can be typed into the editor on the right of the form.

 

You can expand a database table in the treeview to see what fields are available from it.

 

You can insert a table or field into your SQL select statement simply by double clicking on it in the treeview.

If you think the database schema has changed then you can force a refresh of the treeview by right clicking on the database name at the top of the treeview and select Refresh DB Schema.

Once you have defined your SQL select statement click 'Next >' to move to the next screen the Query Wizard - Publish Settings screen. Your syntax will be checked at this point by performing a test execution of the SQL. If there are any errors you will be provided with a description of the error.


Data Definition - Import from a File

 

To Import data from a file, enter the filename you want to use, or select it using the Open File Dialog button.

 

The following file Types are supported.

 

File Types

Descrpition

csv

Comma Delimited

xlsx

Excel Workbook

xls

Excel 97-2003 Workbook

txt

Text

 

clip5861

After selecting the file, click the checkbox if the first row of you file has column headers. Depending on the file type you can then set the Delimiter, use the 'Other' option to set a specific non-standard delimiter.

 

clip5862

 

 

 

Once you have finished making your selections click on the 'Next >' button to go to the Data Definition Wizard - Publish Settings screen.

 


Data Definition - Publish Settings



Once you have finished making your selections click on the 'Next >' button to go to the Data Definition Wizard - Finished screen.



Data Definition - Finish


When you have completed the previous screens, and entered all the information required to create your Data Definition, the Query Wizard - Finished screen will be displayed.

 

clip0085

 

You still have the option to 'Cancel' without saving any of your changes and you can step '< Back' through the previous screens to check your work.

 

To complete your Query click on the 'Finish' button.