1. Use of preferences
Index
-------------------------------------
1. Run ECatalog. Quick launch!
2. Run the User Study. Quick launch!
3. Know more about this package
4. Changing the parameters of ECatalog (use of preferences, trade-off analysis)
5. Improving performance (MORE SPEED!). Using MySQL
6. Using your own database (instead of the provided second-hand cars or apartments)
And tuning the config files
7. The User Study
8. Recompiling
9. License
10. Research work
11. Contact us
1. ECatalog. Quick launch!
----------------------------------
To start the ECatalog, in Ms Windows, execute one of these 2 scripts:
cars.bat
apartments.bat
To start the ECatalog, in Unix, execute one of these 2 scripts:
cars.sh
apartments.sh
2. Run the User Study. Quick launch!
---------------------------------------
If you want to try it right now without reading the details,
print the documents in the folder "userStudyDocuments", and
in Ms Windows, execute this script:
user_study_test.bat
in Unix, execute this script:
user_study_test.sh
Read the printed document titled "Description",
and then follow the instructions on the screen.
To know more about the user study, read section 6: The User Study.
3. More info about this package
----------------------------------
After unziping ecatalog-1.0.zip, you have this directory strucute:
./ecatalog-1.0/readme.html <<This file
./ecatalog-1.0/src <<The source code
./ecatalog-1.0/build <<The class files
./ecatalog-1.0/images <<The icons used in the application
./ecatalog-1.0/databases <<A database of second-hand cars and a database of apartments, in MySQL and Derby format
./ecatalog-1.0/configs <<The config files to run the application, corresponding to the two databases mentioned above
./ecatalog-1.0/logs <<When executing the ECatalog of the User Study software, log files are created here
./ecatalog-1.0/userStudyDocuments <<Documents concerning the user study
./ecatalog-1.0/surveysConfig <<The surveys config files used in the user study
./fileList.html <<A list of all the files.
4. Changing the parameters of ECatalog (use of preferences, trade-off analysis)
---------------------------------------------------------------------------------
To turn on/off these features, you need to edit the config files
(TODO: allow to turn on/off these features from the user interface)
For instance, edit the file configs/configs-cars.xml
To turn off the use of preferences, change this line
<preferencesAllowed>true</preferencesAllowed>
by
<preferencesAllowed>false</preferencesAllowed>
To turn off the use of trade-off analysis, change this line
<conflictAnalysisDisplayed>true</conflictAnalysisDisplayed>
by
<conflictAnalysisDisplayed>false</conflictAnalysisDisplayed>
To know more about tuning the config files, read
section 6: Using your own database and tuning the config files
5. Improving performance (MORE SPEED!). Using MySQL
------------------------------------------------------
In this package, we provide a database of second-hand cars and a database of apartments,
in MySQL and Derby format.
Derby is an embedded SQL database fully implemented in Java.
As it is embedded, you do not need to install anything.
On the other hand, as it is implemented in Java, it is less performant than other implementations.
MySQL is a SQL database implemented in C, and very performant.
As it is not embedded, you need to install the software and databases yourself,
which it is quite annoying for common users.
On the other hand, it is much more performant than Derby.
If you want that ECatalog runs faster, you need to install MySQL.
[TODO: propose also to use MsAccess]
Installing the databases for MySQL:
Download and install MySQL 5.0.
[http://dev.mysql.com/downloads/mysql/5.0.html]
Open a console and execute:
> mysql -uroot -p
Enter password:
mysql> CREATE DATABASE test
mysql> GRANT ALL ON test.* TO 'mylogin'@'localhost' IDENTIFIED BY 'mypwd';
mysql> GRANT ALL ON test.* TO 'mylogin'@'yourmachine.com' IDENTIFIED BY 'mypwd';
mysql> EXIT
> mysql -umylogin -pmypwd test <databases/mysql/car_car5.sql
> mysql -umylogin -pmypwd test <databases/mysql/appartement.sql
> mysql -umylogin -pmypwd test <databases/mysql/timeFromZipCode.sql
> mysql -umylogin -pmypwd test
mysql> SELECT COUNT(*) from car_car5;
[it should say 200]
mysql> SELECT COUNT(*) from appartement;
[it should say 186]
mysql> SELECT COUNT(*) from timeFromZipCode;
[it should say 1000]
mysql> EXIT
Now, MODIFY the config files to use MySQL instead of Derby:
For instance, edit the file configs/configs-cars.xml
replace:
<DBURL>jdbc:derby:databases/derby/testdb;create=false</DBURL>
by
<DBURL>jdbc:mysql://localhost:3306/test?user=mylogin&password=mypwd</DBURL>
Great! Now it should run much faster!
[TODO: Still, sometimes it takes considering time to compute the Trade-off Analysis.
Optimize this phase, precompile this information...]
6. Using your own database (instead of the provided second-hand cars or apartments)
And tuning the config files
-------------------------------------------------------------------------------------
You can use Apache Derby or MySQL.
[TODO: propose also MsAccess]
To import your database into Derby:
Create a SQL file similar to ./databases/derby/car_car5.sql
REMEMBER to create indexes (see the bottom of the car_car5.sql file),
otherwise ECatalog will run really slowly!
Download and install Apache Derby (We used Derby 10.1.3.1)
http://db.apache.org/derby/
db-derby-10.1.3.1-bin/frameworks/embedded/bin/ij.ksh <yourdatabase.sql
Derby will create a directory (in the example is called "testdb").
Put this directory in ./databases/derby/
Once you have your database (in MySQL or Derby), you need to create a config file.
For the moment, look at the file "configs/config-apartments.xml",
and modify it according to your database.
Sorry for this, we are working to write about this.
Write to us if you have problems. We'll do our best to answer you promptly.
[TODO: create a configuration tool. or maybe from the same ECatalog software]
Once you have created your config file, to run ECatalog, execute:
in Ms Windows:
> env.bat
> java ecatalog.ECatalog configs/your_config.xml
in Unix:
> . env.sh
> java ecatalog.ECatalog configs/your_config.xml
7. The User Study
-------------------------
Note: you can download the log files of our user study!
7.1 INTRODUCTION
------------------------------------------------------------
We have two dependent variables:
1. W [Yes/No]: Use of preferences (Weights)
2. T [Yes/No]: Use of Trade-off analysis
So, we have 4 different interfaces:
0- W no, T no
1- W yes, T no
2- W no, T yes
3- W yes, T yes
We make 3 groups of 10 people.
Group 1 will use Interface 0, and messure the improvement of using Interface 1
Group 2 will use Interface 0, and messure the improvement of using Interface 2
Group 3 will use Interface 0, and messure the improvement of using Interface 3
7.2 PERFORMING THE USER STUDY
-------------------------------------------------------------
Advice: Follow section 5: Improving performance (MORE SPEED!). Using MySQL
before starting the real experiments!
Divide the 30 participants in 3 groups of 10 people, randomnly.
Print the following documents:
-Description.doc
-Consent Form.doc
-Tutorial_baseline.doc
-cars-attributes.doc
-Tutorial_improved_w.doc
-Tutorial_improved_t.doc
-Tutorial_improved_wt.doc
7.2.1 For each participant:
------------------------------
Start the User Study software:
in Ms Windows:
> user_study.bat <participantId> <interfaceId>
in Unix:
> user_study.sh <participantId> <interfaceId>
For instance:
> user_study.bat participant16 3
Give him the documents:
-Description.doc
-Consent Form.doc
-Tutorial_baseline.doc
-cars-attributes.doc
And depending on his group, also the following one:
1- Tutorial_improved_w.doc
2- Tutorial_improved_t.doc
3- Tutorial_improved_wt.doc
Read to him the document titled "Description",
and explain him the Timeline.
Ask him to follow the instructions on the screen.
After the experiment, His log files + surveys will be saved in ./logs/<participantId>/
7.4 AFTER ALL THE 30 EXPERIMENTS
We will have these messures:
- Decision time: I do not expect to be able to use this messure, because all users will take around 5min
- Decision accuracy: Did the user really find his best possible choice? Validation phase (yet not extremely convincing)
this is cumbersome. the validation phase is not very good, but it is an "accepted" messure in the community.
[TODO: find a better idea to messure decision accuracy]
From the surveys:
- Confidence: Is the user sure that he found his best possible choice (from the ones present in the database)?
- User cognitive load (NASA TLX), time, enjoyableness
- Perceived Usability - Usefulness
- Enjoyable: Did he get frustrated when using the system?
8. Recompiling
-------------------------------------
Look at the file ./Makefile
To restart from scratch:
>make clean
If you want to modify the XML schemas:
>make xml
You will need to install jwsdp-2.0 if you want to change the XML schemas
http://java.sun.com/webservices/downloads/webservicespack.html
To compile the software:
>make cmp
9. License
-----------------------------------------
ECatalog is a database front-end, with two main features:
1. Use of preferences
A preference-based approach, where the user is allowed to define the importance of each criterion.
Then the items are ranked accordingly to his criteria.
2. Trade-off analysis
A cooperative database approach, where the system "argues" with the user about his criteria.
When there are no matching items, the system explains the minimal conflicting set and give some possible strong and weak relaxations about his criteria.
This package also containts the software and the set-up details used for our User Study,
comparing the use or not of the two previous features mentioned above.
Copyright (C) 2006 David Portabella Clotet, Artificial Intelligence Laboratory, EPFL
This file is part of ecatalog-1.0.zip
ECatalog is free software and a free user study set-up;
you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
ECatalog is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with ECatalog; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
@version 1.0
@author David Portabella
To contact the author:
email: david@portabella.name and david.portabella@epfl.ch
More information about ECatalog:
http://sourceforge.net/projects/ecatalog/
http://david.portabella.name/ecatalogs/
---
The database of second-hand cars is contributed from comparis.ch for the sake of research only.
If you use this database, Comparis.ch would appreciate if you send a short mail to them,
Johann Burkhard <johann.burkhard@comparis.ch>, stating your email and an one/two-sentence description of the use.
10. Research work
---------------------------------------------------------------------------------
Our research related to ECatalog:
Improving User Confidence in Decision Support Systems for Electronic Catalogs.
Download
More info
If you use the software and/or the user study provided, we would appreciate if you
send us a short mail to David Portabella <david.portabella@gmail.com>,
stating your email and an one/two-sentence description of the use.
If you would like to cite our work, please you this citation:
David Portabella. Improving User Confidence in Decision Support Systems for
Electronic Catalogs. Phd. thesis no. 3737, Swiss Federal Institute of
Technology (EPFL), Lausanne (Switzerland), 2007.
@PhdThesis{Portabella2007thesis,
author = {Portabella, David},
title = {Improving User Confidence in Decision Support Systems for Electronic Catalogs},
school = {Swiss Federal Institute of Technology (EPFL)},
year = {2007},
address = {Lausanne (Switzerland)},
type = {PhD. Thesis No. 3737},
bibtex_key = {Portabella2007thesis},
url = {http://david.portabella.name/files/EPFL_TH3737.pdf}
}
Take-Away message:
Conflict and corrective feedback are the most promising types of feedback for decision
confidence building. This claim is supported by experimental evaluation in the case of
ranked list feedback compared to conflict and corrective feedback.
In typical situations where there is a mismatch between what the decision maker wants and
what is it in the catalog, Confidence Building Tradeoff Explanations (CBTE) provide a clear
trade-off question to help the user make a good decision through the global relaxation set,
and provides evidence for building decision confidence through the minimal conflict set and
the local relaxation set.
In the same experimental evaluation, we saw that CBTE system feedback incited more
participants to make more challenging queries, and thus to state more accurate objectives
rather than mean objectives. This is a very beneficial behavior because DSSECs can better
help them if users state precise objectives.
---
The database of second-hand cars is contributed from comparis.ch for the sake of research only.
If you use this database, Comparis.ch would appreciate if you send a short mail to them,
Johann Burkhard <johann.burkhard@comparis.ch>, stating your email and an one/two-sentence description of the use.
11. Contact us
-----------------------------------------
David Portabella
Email: david@portabella.name
Email: david.portabella@epfl.ch
Last modified: 15th July 2006