Differences

This shows you the differences between two versions of the page.

--- gt3:loads:import:defined:bulkhandler [18/05/2012 03:35]
Ian Davidson [Matching Suppliers]
+++ gt3:loads:import:defined:bulkhandler [20/08/2018 04:24] (current)
@@ Line 3: / Line 3: @@
 {{indexmenu_n>3}}
 <note>This documentation applies to Grain Trader version 3.3.5 and later. An earlier implementation of this functionality, which facilitated the import of GrainCorp Acquisition and Payrun files only, was present in all version 3.0.0 and later versions but has been consolidated into this implementation.</note>
-You can import load data directly from an external file sourced from a bulk handler. At the time of writing the following formats are supported:
+You can import load data directly from an external file, either sourced from a bulk handler or in Transport mode from another Grain Trader database. Before importing you need to configure your Grain Trader system so that it can recognise the codes and identifiers used by the bulk handler and if applicable the Grain Trader trading database. This is covered fully in the sections [[gt3:loads:import:defined:gtimportsetup]] and (if applicable) [[gt3:loads:import:defined:gtdatabase]].
+At the time of writing the following bulk handler formats are supported:
   * GrainCorp Acquisition File
   * GrainCorp Payrun File
@@ Line 12: / Line 14: @@
-====Matching Commodities====
-Commodities are matched using the Cross Ref code entered on the Commodity window:
-{{ :gt3:loads:import:commodxref.png?nolink |}}
-It is assumed that all bulk handlers use the same code to identify the same commodity and grade eg APW for APW Wheat.
-====Matching Storages====
+===== Importing the Loads =====
-Each Bulk Handler will have their own codes for the storage sites used. Our recommendation is that you create a storage for each Commodity and Bulk Handler you use; this will make it easier for you to keep track of who owns what and where over time. If later you no longer wish to use a combination you can just make that storage Inactive and it won't be available for selection from drop-down lists etc.
+To import loads from your Bulk Handler file, select the Utility | Import | Loads From Defined Format File menu:
-You need to obtain the Storage cross-reference codes used by the Bulk Handler you are dealing with from them, and enter them in the XRef column of the grid on the Storages tab of the Commodity window, as shown below:
-{{ :gt3:loads:import:storagexref.png?nolink |}}
-====Matching Metrics====
-//Metrics// are the measurements of the various parameters of the grain that can be used to determine its grade and/or quality and/or price - for example, Protein, Moisture, Oil Content etc. You might not be using them for purposes such as these, and depending on the grain, the Bulk Handler and other factors they may or may not be present in the import file you obtain from the Bulk Handler.
-Even if you're not using metrics to classify the grade or make bonuses or deductions based on their values, you may wish to record them for potential use in the future. Grain Trader recalculates a weighted average of up to the first five metrics for the commodity in each storage on every incoming load, so you can tell at a glance the overall storage averages.
-<note>If you do not wish to record values for any individual or all metrics (and they are included in the Bulk Handler's import file), you need not configure the Commodity for those metrics (as described in the section [[bulkhandler#Defining the XRef Term For a Commodity and Handler|Defining the XRef Term For a Commodity and Handler]] below). However any metrics found in the import file which are not cross-referenced in the Grain Trader commodity will generate an entry in the [[bulkhandler#The Import Error Log|import error log]] unless you choose to ignore them - see the section [[bulkhandler#Importing the Loads|Importing the Loads]] below for more detail. </note>
-===Configuring the Bulk Handler Schedule===
-Because there are different import file types available from some Bulk Handlers (notably GrainCorp), but all the formats from one handler can use the same cross-reference terms, Grain Trader allows you to define the terms only once for each Bulk Handler and Commodity (however the system is flexible enough to allow for a different list of terms should this be needed in the future). To configure these you first need to give each Bulk Handler schedule of terms a name of your own choosing, then allocate the file format(s) they provide to their schedule. Select the //Utility | Import | Manage Load Import Formats// menu to display the **Manage Load Import Formats** window:
-{{ :gt3:loads:import:importmanagemenu.png?nolink |}}
-The names you give to Schedules are used as the column headings on the Bonuses & Deductions grid on the Commodity window. The order of the Schedule names on the Manage Load Import Formats window top-to-bottom corresponds to the order left-to-right on the Commodity window.
-To add a new Schedule, click //New// and enter the name you want to use in the grid, then click //Apply//.
-{{ :gt3:loads:import:importmanageschedule.png?nolink |}}
-You can also delete a Schedule if it is no longer required. Deleting a Schedule removes the column with that heading from all Commodity windows, and all its XRef values from the database. Existing loads are not affected in any way, but you will not be able to import additional loads using that Bulk Handler Schedule.
-You then need to associate the import File Format(s) to its Schedule as appropriate. CDA Software provides a file for each type of Bulk Handler import file, whose contents define how Grain Trader is to interpret the import file. These definition files have an extension of ''.idf'' and reside in the same folder as your Grain Trader program, usually
-''C:\Program Files\CDA Software\CDA Grain Trader''
-on 32-bit systems or
-''C:\Program Files (x86)\CDA Software\CDA Grain Trader''
-on 64-bit systems. Grain Trader automatically finds these files if they exist on your computer (or terminal server) and stores information about them in your database. If the file is later deleted from disk, the stored data is not automatically removed, but the checkbox in the Found column of the Formats grid will be unticked, and you will not be able to import files of this type from the workstation you are using. Definition files for all currently-supported formats are installed when you install Grain Trader or a patch upgrade. You can download additional or missing formats from the CDA Software website if needed.
-<note tip>Contact CDA Support if you need to be able to import files of a format that is not currently supported. You will need to send us a copy of your Grain Trader database, sample import file(s) and a definition of the import file contents (usually obtained by the Bulk Handler). (We may charge you for this work but if so will discuss this with you prior to any work being carried out.)</note>
-To associate an import File Format with a Bulk Handler Schedule, click in the //Metrics XRef Schedule// column cell of the Formats grid for the Format you want to associate and select the Schedule from the drop-down list, then click //Apply//.
-{{ :gt3:loads:import:importmanageformats.png?nolink |}}
-You can also delete a format definition (.idf) file from your disk, and all its associated data from your database, by clicking //Delete//. Existing loads are not affected in any way, but you will not be able to import additional loads using that file format. Note however that if the format is still supported by Grain Trader, the next time you install a patch update the format definition file will be re-installed, automatically found and added again.
-===Defining the XRef Term For a Commodity and Handler===
+{{ :gt3:loads:import:importmenu.png?nolink |}}
-When you have finished configuring the Bulk Handler import formats you can enter the Bulk Handler's cross-reference codes for each metric on the Commodity window. Scroll the Bonuses & Deductions grid across to the right to see the column(s) for each Schedule you have defined, and enter the values in the cells for each Bulk Handler and Metric as required.
-{{ :gt3:loads:import:commodmetrics.png?nolink |}}
-<note>If you do not enter a cross-reference code for a metric and later import a Bulk Handler file that contains this metric it will not be imported and this fact may be recorded in the [[bulkhandler#The Import Error Log|import error log]], depending on your other import settings. See [[bulkhandler#Importing the Loads]] below for more details.</note>
-===== Importing the Loads =====
-To import loads from your Bulk Handler file, select the Utility | Import | Loads From Bulk Handler File menu:
-{{ :gt3:loads:import:importloadsmenu.png?nolink |}}
 The **Import Loads From Bulk Handler File** window will be displayed.
 {{ :gt3:loads:import:importloads.png?nolink |}}
-Click the Browse button to locate and select the Bulk Handler file you want to import, and select the Import File Format from the drop-down box.
+Click the Browse button to locate and select the Defined Format file containing the loads you want to import, and select the Import File Format from the drop-down box.
 ==== Import Options ====
@@ Line 76: / Line 40: @@
   * Click //Cancel// to abort the import altogether; for example you might not have realised this file has previously been imported.
-In any event a record noting the duplicate and your action will be saved in the [[bulk#The Import Error Log|import error log]].
+In any event a record noting the duplicate and your action will be saved in the [[bulkhandler#Managing Import Log Files|import log file]].
 If you select the //Use the next auto Load Number// option you'll not be prompted about this because Grain Trader ensures a Load Number is not duplicated.
@@ Line 86: / Line 50: @@
   * Click //Yes// to save the load with the same Ticket Number. Select this option only if you are certain this load has not been imported before; the duplicate Ticket Number must just be a coincidence. Note that if you have also selected to use the Ticket Number as the Load Number as described above, and a load with that Load Number already exists you will be prompted again about the duplicate Load Number, as described above.
-  * Click //No// to skip this load and continue; for example you've imported this file before, have corrected some errors and are now re-importing it. A record noting the duplicate and your action will be saved in the [[bulk#The Import Error Log|import error log]].
+  * Click //No// to skip this load and continue; for example you've imported this file before, have corrected some errors and are now re-importing it. A record noting the duplicate and your action will be saved in the [[bulkhandler#Managing Import Log Files|import log file]].
-  * Click //Cancel// to abort the import altogether; for example you might not have realised this file has previously been imported. A record noting the duplicate and your action will be saved in the [[bulk#The Import Error Log|import error log]].
+  * Click //Cancel// to abort the import altogether; for example you might not have realised this file has previously been imported. A record noting the duplicate and your action will be saved in the [[bulkhandler#Managing Import Log Files|import log file]].
-If you select the //Don't import the load; just log it and skip to the next one// option the record will just be skipped; for example you've imported this file before, have corrected some errors and are now re-importing it. A record noting the duplicate and your action will be saved in the [[bulk#The Import Error Log|import error log]].
+If you select the //Don't import the load; just log it and skip to the next one// option the record will just be skipped; for example you've imported this file before, have corrected some errors and are now re-importing it. A record noting the duplicate and your action will be saved in the [[bulkhandler#Managing Import Log Files|import log file]].
 ===Contract Mismatch Options===
 If you select the //Prompt me to choose what action to take// option and a load with a Contract is being imported, and whose imported commodity is not one of the Grain Trader Commodities on the Contract, you will be prompted as shown below:
 {{ :gt3:loads:import:importloadscommodcontract.png?nolink |}}
 Take one of the following actions:
-  * Click //Yes// to save the load and use this contract anyway. This is the equivalent of clicking the //Show All// button on the Load window in Grain Trader and selecting a Contract with a different Commodity on it. The [[bulk#The Import Error Log|import error log]] record will include a notation that you should check the details for this load.
+  * Click //Yes// to save the load and use this contract anyway. This is the equivalent of clicking the //Show All// button on the Load window in Grain Trader and selecting a Contract with a different Commodity on it. The [[bulkhandler#Managing Import Log Files|import log file]] record will include a notation that you should check the details for this load.
   * Click //No// to import the load but ignore the imported contract, leaving the Grain Trader Load without a Contract.
   * Click //Cancel// to skip the load altogether.
-In any event the details of the load and your action will be recorded in the [[bulk#The Import Error Log|import error log]].
+In any event the details of the load and your action will be recorded in the [[bulkhandler#Managing Import Log Files|import log file]].
 The remaining three options on this panel have the same actions as the three responses above, except that the action will be taken automatically and no prompt will be raised.
 ===Ignore Metrics without matching XRefs in Grain Trader===
-If a load being imported contains data for a Metric for which you have not recorded a cross-reference value as described in the [[bulk#matching metrics|Matching Metrics]] section above, this checkbox governs the program's action.
+If a load being imported contains data for a Metric for which you have not recorded a cross-reference value as described in [[gt3:loads:import:defined:gtimportsetup#matching metrics|Matching Metrics]], this checkbox governs the program's action.
-  * If the checkbox is unticked an record will be stored in the [[bulk#The Import Error Log|import error log]].
+  * If the checkbox is unticked a record will be stored in the [[bulkhandler#Managing Import Log Files|import log file]].
   * If the checkbox is ticked, no error will be logged and the data will be ignored.
@@ Line 116: / Line 82: @@
 When the import has finished, either normally or abnormally because you aborted the process, a dialog similar to the following will be displayed:
-{{ :gt3:loads:import:importfinished.png?nolink |}}
+{{ :gt3:loads:import:importfinishedwitherrors.png?nolink |}}
-If no errors or warnings have been logged the dialog will just advise you of the number of records imported successfully; no error log will have been created. Otherwise the dialog will also advise you how many warnings or errors were logged and prompt you to review the log file straight away, as shown in the sample image above. You should always review the log if it exists to ensure you haven't missed importing any loads - for example if a Commodity, Supplier or Storage could not be matched to the import data. You can also review and manage log files later as described in the [[bulk#The Import Error Log]] section below.
+If no errors or warnings have been logged the dialog will just advise you of the number of records imported successfully. A log file will still have been created; it will contain the basic details such as the name of the imported file and number of loads successfully imported, so that you can review it later to see what was done. Otherwise the dialog will also advise you how many warnings or errors were logged and prompt you to review the log file straight away, as shown in the sample image above. You should always review the log if it exists to ensure you haven't missed importing any loads - for example if a Commodity, Supplier or Storage could not be matched to the import data. You can also review and manage log files later as described in the [[bulkhandler#Managing Import Log Files]] section below.
-If you find errors such as missing NGR numbers or cross-references, you can fix these issues in Grain Trader then re-import the same file, making sure that the [[bulk#Import Options|import options]] you have selected and/or responses to prompts will avoid duplicating loads.
+If you find errors such as missing NGR numbers or cross-references, you can fix these issues in Grain Trader then re-import the same file, making sure that the [[bulkhandler#Import Options|import options]] you have selected and/or responses to prompts will avoid duplicating loads.
 You should also be sure to check any loads whose contract was changed, to make sure the correct pricing has been applied to the commodity imported.
+If there were no errors or warnings the Defined Format file that contained the loads just imported will be moved from its original location to a folder beneath the folder where your Grain Trader database resides, ie:
+''{your database folder}\Archived Import Files''
+so as to make it more difficult to accidentally re-import the same file. If errors or warnings occurred the file will not be moved. You can manage the archived Import files later as described in the [[bulkhandler#Managing Archived Import Files]] section below.
 All imported loads are created as saved loads, and you can review and change them before processing them as normal.
-==== The Import Error Log ====
+==== Managing Import Log Files ====
-As mentioned a number of times in the documentation above, information about various errors, warnings and actions taken during the import process is saved to a log file. These files are located in the same folder as the Grain Trader database into which the loads are being imported and are named according to the following format: {Database Name} Bulk Handler Import {yyyy-mm-dd}.log where {Database Name} is the name part of your database (.mdb) file, without the .mdb, and {yyyy-mm-dd} represents the date. It follows that one log file will be created for each combination of a database into which you import loads and the date on which you do the import. Multiple imports on one day into the same database just keep appending information to the same log file. Each import session is clearly defined and includes details such as the file being imported, the date and time and the selected import file format.
+As mentioned a number of times in the documentation above, information about the Defined Format File being imported, together with various errors, warnings and actions taken during the import process is saved to a log file. These files are located in a folder beneath the folder where the Grain Trader database into which the loads are being imported resides, ie:
-You should always review the error log if one is generated by your import, to ensure you haven't missed any loads (for example, due to NGR numbers, commodities or storages that can't be matched to your Grain Trader equivalents). Similarly, even if loads have been imported there might be missing Metric data, different Contract numbers and so on.
+''{your database folder}\Import Log Files''
-You can either review an error log if one exists on completion of the import process by clicking OK at the prompt, as described in [[bulk#Finishing the Import]] above, or from the Error Logs tab on the **Import Loads From Bulk Handler File** window.
+and are named according to the following format: {Database Name} Defined Format Import {yyyy-mm-dd}.log where {Database Name} is the name part of your database (.mdb) file, without the .mdb, and {yyyy-mm-dd} represents the date. It follows that one log file will be created for each combination of a database into which you import loads and the date on which you do the import. Multiple imports on one day into the same database just keep appending information to the same log file. Each import session is clearly defined and includes details such as the file being imported, the date and time and the selected import file format.
-{{ :gt3:loads:import:importloadsmanagelogs.png?nolink |}}
+You should always review the log file if errors or warnings occurred during your import, to ensure you haven't missed any loads (for example, due to NGR numbers, commodities or storages that can't be matched to your Grain Trader equivalents). Similarly, even if loads have been imported there might be missing Metric data, different Contract numbers and so on.
-The default folder being viewed is that in which the database that is currently open in Grain Trader resides, which is the same folder that log files relating to imports into this database will be created. However if you have multiple databases in different folders you can also use the browse (...) button to select a different folder if needed to manage the log files (if any) in that location.
+You can either review the log file on completion of the import process by clicking OK at the prompt, as described in [[bulkhandler#Finishing the Import]] above, or from the Log Files tab on the **Import Loads From Defined Format File** window.
+{{ :gt3:loads:import:importmanagelogs.png?nolink |}}
+The default folder being viewed is that in which log files relating to imports into this database will be created. However if you have multiple databases in different folders you can also use the browse (...) button to select a different folder if needed to manage the log files (if any) in that location.
 Select one or more rows in the grid and click either //View// to review the log file in your plain text editor (eg Notepad), or //Delete// to delete them.
+==== Managing Archived Import Files ====
+As mentioned above, when you have imported a Defined Format file without errors or warnings it will be moved into the archive folder to make it easier to avoid re-importing the same file at a later date. This folder is located beneath that in which the Grain Trader database into which the loads are being imported resides, ie:
+''{your database folder}\Archived Import Files''
+You can manage these archived files from the Archived Import Files tab on the **Import Loads From Defined Format File** window.
+{{ :gt3:loads:import:importmanagearchived.png?nolink |}}
+The default folder being viewed is that into which the import files relating to imports into this database will be moved. However if you have multiple databases in different folders you can also use the browse (...) button to select a different folder if needed to manage the archived import files (if any) in that location.
+Select one or more rows in the grid and click //Delete// to delete them.