Sforce Data Loader: Using the command line interface
by Andrew Waite on June 13, 2005 at 03:13 PM
The Sforce Data Loader can be scripted to run from a command line by following these steps:
1. Prepare your system
To use the Command Line Interface ("CLI") of the Sforce Data Loader your target machine (the machine upon which the loader will run via the CLI) needs to have the Java Runtime Environment ("JRE") v1.4.2_03 or greater installed and configured. JRE 1.5 is not supported.
The CLI of the Sforce Data Loader uses the same processing engine as the Graphical User Interface ("GUI") provided in the Windows installation.
In order to use the CLI you must first install the Sforce Data Loader on your Windows computer using the .exe provided in setup of Salesforce.
If you intend to use the CLI of the Sforce Data Loader on the same machine on which it is installed you need not move any files. In this case you can skip to "3. Setting operation parameters" below.
If the target machine for your scheduled process is not the same as where you installed the GUI using the .exe then you need to perform step 2.
2. Copy the loader engine and files
Once the GUI is installed, open the directory you selected during install, the default is:
C:\Program Files\salesforce.com\Sforce Data Loader
Copy the sforcedataloader.jar file located in this directory the working directory of your batch process on the target machine. In this working directory, create a folder named "conf" and copy everything in C:\Program Files\salesforce.com\Sforce Data Loader\conf to this directory.
Alternatively, you can download SforceDataLoader.zip which contains the jar file and conf directory for those that can't easily install the windows gui.
3. Setting operation parameters:
Depending on your operation you must provide a minimum set of parameters to run. These parameters can be specified in either the config.properties file in your conf directory or as command line arguments. Modify the parameters according to your operation and desired behavior. Sample required parameters for a basic extract operation of Account records are as follows:
# Required parameters for processing an Extract operation
username=you@yourcompany.com
password=yourpassword
operation=extract
extractionSOQL=SELECT Id,Name FROM Account
extractionTarget=account-extract.csv
Insert/Update/Delete operations have a separate set of required parameters. A sample of the required parameters for a basic input operation for Account records are as follows:
# Required parameters for processing an Insert operation
username=you@yourcompany.com
password=yourpassword
operation=inserts
batchSize=200
mappingFile=conf/default.sdl
dataAccessObject=account-insert.csv
entity=Account
outputSuccess=success.csv
outputError=error.csv
Best Practice: Run the process once manually using the GUI, specifying the parameters as desired. The settings from the manual execution will be saved to:
C:\Documents and Settings\<windows username>\Application Data\salesforce.com\Sforce Data Loader\config.properties
If the operation is not an extract, be sure to save your map as well('default.sdl' above).
Copy that config.properties file and your map (.sdl) to the conf directory in the working directory of your target machine.
Modify the values in config.properties to contain valid system paths for the appropriate files on your target system (the examples here assume a linux/unix system).
4. Prepare and execute
Refer to the section titled, "Using the Command Line Interface" in the Sforce Data Loader User Guide for detailed information to prepare the command and run the data loader from the command line.
After the load is completed, the program will export your parameters to the config.properties file. This process will not include any comments. If you want to maintain your properties for reference, save them under a different file name. Note: this resave will not include the password property.
Best Practice: If your process is repetitive you should be passing the password as a command-line argument.
You should also pass in the outputSuccess and outputError parameters when performing inserts/updates/deletes and extractionTarget when performing extract as the loader will overwrite files found to have the same filename.
5. Parameter Reference
####### BEGIN FILE ############################
# Loader Config Parameters
#
# This can be copied as your sample config.properties file. Lines
# beginning with a '#' indicate comments and will be ignored by the
# program.
###### Required for All operations ####################
#
# Salesforce user account credentials. Your account must be active
# and you will see better performance if you run as a System
# Administrator
username=you@yourcompany.com
password=yourpassword
# The operation type being performed. A valid value is
# 1 of: extract, inserts, updates, deletes
operation=extract
#
####### Required only for extract ###################
#
# The query that defines the scope and type of data to be returned
# from sforce. For more information on the Sforce Object Query Language
# please see the API documentation on sforce.com
extractionSOQL=SELECT Id,Name FROM Account
# The file that will contain the data extracted from Salesforce
extractionTarget=account-extract.csv
#
####### Required only for Inserts/Updates/Deletes ###########
#
# The path to the file that contains the field mapping for your
# operation. It is recommended that you use the Sforce Data Loader
# GUI to generate the mapping file. Note: you must map the Id field in
# updates/deletes operations and you must NOT do so in inserts
mappingFile=conf/default.sdl
# The path to the file containing the source data
dataAccessObject=account-update.csv
# The name of the entity that is the target of the data. This is
# any accessible object in the API. Note: if this is a custom object
# you must append the '__c' suffix (two under-scores, then the letter c)
entity=Account
# The path to the file that contains the success output
outputSuccess=success.csv
# The path to the file that contains the error output
outputError=error.csv
# The size (number of records) of each request
batchSize=200
#
####### Optional parameters #####################
#
# Please refer to the Sforce Data Loader User Guide for more
# information regarding these parameters (default values are shown)
#endpoint=https://www.salesforce.com
#lastBatchRow=0
#assignmentRule=
#rowToStartAt=0
#extractionRequestSize=2000
#insertNulls=false
#timeoutSecs=60
#proxyPassword=
#proxyHost=
#proxyPort=
#
###### END OF FILE #######################

Comments
Posted by Basil Karam on June 16, 2005 09:27 PM:
When I execute the command prompt to extract data, the command window returns 1000's of debug statements. How can I turn this off
Posted by Andrew Waite on June 16, 2005 10:19 PM:
Thanks for the comment Basil. Make sure to take note of the information in the User Guide per step 4., above.
If you set the salesforce.config.dir variable and the directory you supply has the log-conf.xml file in it you should not get this level of logging.
Happy Loading!
Posted by Cecile Franchini on June 27, 2005 07:35 AM:
Is there a way to encrypt the password?
Posted by Rup on June 28, 2005 11:17 AM:
OK, how does this work on Mac OSX (Tiger 10.4.1, Java 1.4.2_07) ?
Posted by Andrew Waite on June 28, 2005 05:14 PM:
This will test the promise of java :) you should be able to follow the same instructions per above. If you don't have access to a windows machine to install the loader and get the necessary files I will update the blog with a zip file containing what you need.
Posted by Tom Greenbank on July 6, 2005 03:47 AM:
For the record this works fine on Linux.
Posted by Roland B on July 13, 2005 08:17 AM:
I'm trying to figure out how to schedule a load/extract jobs within the Sforce data loader
Posted by Andrew Waite on July 13, 2005 08:36 AM:
Roland, you will need to write a script (.sh or .bat) file and then have a system scheduler call it. There is no scheduling mechanism inside the loader itself.
Posted by Pierre Eymard on July 22, 2005 09:42 AM:
Are you planning on updating the blog page, for instance by completing the explanation of all the parameters in conf.properties ? Also, could you make clearer (in red) that the operations are named insertS, updateS,.. and write a paragraph for the required parameters for deletes and updates ? Thanks.
Posted by Narayana on August 3, 2005 04:50 PM:
Hi,
When I try to use CLI of the sforcedata loader I am getting the error as listed below:-
Exception in thread "main" java.util.zip.ZipException: The system cannot find the file specified
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(Unknown Source)
at java.util.jar.JarFile.(Unknown Source)
at java.util.jar.JarFile.(Unknown Source)
The comments to this entry are closed.