HYOWON LEE  B.Eng., M.Sc., Ph.D.
click to visit SUTD home page  
Designer-Researcher at Singapore University of Technology and Design
   Research   Publications   Teaching   C.V.  

In 2005, I have worked on many things, mostly designing interfaces and writing papers on them, paper reviews and helping organising ESSIR2005 event. The timeline below shows (omitting many other smaller tasks), a few mainstream, heavy-weight tasks that required my continuous workload throughout (I see 4 lines), and there were more separate, isolated tasks.


Paper review for ECIR 2006 (December)
As a Programme Committee, I am reviewing papers for ECIR 2006 - 28th European Conference on Information Retrieval (Imperial College London, 10-12 April 2006).


TRECVid 2005 Gaithersburg, Maryland (14-15 November)
I attended TRECVid 2005 in Gaithersburg, with Colulm, Sinéad, Markus and Alan. I helped the portable Físchlár-DT system for demo session and the poster.


AIC Conference, Dublin (4 November)
I participated in the annual AIC Conference at Crowne Plaza Hotel, Dublin. Colum, Sinéad and I together presented a poster on our Físchlár-DT system and its experiment.


Interface Design for BBC Rushes Exploration (November)
I drove the user-interface side of the BBC Rushes exploration task for TRECVid05 in our Centre. Starting with PowerPoint-based prototype that illustrates initial interaction sequence, I refined it based on our meeting outcome (Alan, Gareth, Noel, Sorin, myself, Colum, Sinéad, Aiden and Kevin) and finalised in Photoshop, which Sorin took and implemented into a working system. The interface allows its user to find example images from Google Image or any other image search engines on the web, and bring those into the interface either the whole image itself or segmented objects only. Once incorporated into the interface, the user can use those as relevance feedback image to find similar images or objects within the BBC rushes database. Using the object highlighting scheme that I have developed for my previous object-based interfaces, dealing with a whole keyframe or object within it is easy and intuitive. This work is briefly presented at TRECVid2005 workshop in Gaithersburg, Maryland.


Tabletop Interaction: design - implementation - evaluation (April - October)
Colum, Sinéad and I are working on exploring collaborative interaction for video searching, with tabletop systems. We are using MERL's DiamondTouch table, and have already designed and implemented various versions of video searching applications we call Físchlár-DT. We have identified important design considerations for tabletops: labour division among the searchers, workspace awareness and software coordination policy. Each of these are the dimensions where we design suitable interfaces with different options in each, and we implement and evaluate them. Geared to TRECVid 2005 Search task, we have conducted an experiment with 8 pairs of users (16 users), with CCTV recording of the whole sessions (including users' gestures and voice). We analyse the users' collaboration levels and conflicts to measure how our designed tabletop interfaces support collaboration.


ESSIR 2005 (5-9 September)
Hosting European Summer School in Information Retrieval this year, I was involved in various organisation activities, running around here and there during the full 1-week event. The event successfully completed - see some photos during the event.


MediAssist Web Interface Design
Neil, Cathal and I worked together on re-vamping the MediAssist web system - to make it look better (visceral and reflective level) and to facilitate better browsing by feature indications, annotation and other navigation elements (behavioural level). Icons for weather and light status have been designed and put around each photo. Log-in facility has been added, and User Guide page was also designed.


CoLIS5 Workshop, Glasgow (8 June)
I went to Glasgow at a CoLIS5 Workshop on Evaluating User Studies in Information Access, to present my work on the detailed methodology I took for the Físchlár-News user trial of Spring last year. 30 people gathered, my presentation was the first presentation in the morning. While the detailed findings of this user trial is written for somewhere else, in this talk I focused on the procedure and methodology I took, especially how I conducted the 1-month-long trial with 16 participants by monitoring their usage with interaction logging and asking them to keep a diary. I received very useful feedback, and also the discussion on the issues related to user evaluation was very interesting and useful, including whether we should pay the users, training them, separating usability problems from the system under evaluation, as well as gathering some exemplary work for our own sake.


User-Interface to Bart's Event detection application (May)
I've been discussing with Bart on what the best way would be to provide front-end for his automatic event detection algorithms (for action, dialogue and montage) for films.... we've done a few sketches together and came up with Photoshop version of the interface. Bart implemented in Visual Basic, and it was used for user experiment in October.


Tabletop interface study (April - )
While I have been designing user-interfaces to video systems mainly on desktop PCs (with keyboard, mouse and comfortable TV size screen) some amount on the palmtops (no keyboard, touch screen, limited screen space, distracting environment), the design for the tabletop is new to me. I started reading around tabletop interface concerns... this extends my usual scope of design concerns, to include the social aspects of interaction. Orientation of objects and territoriality is very important consideration on tabletop, and you can see the collaborators say very often "sorry" to each other during the interaction. With the DiamondTouch tabletop system in our centre, I am starting to experiment ideas and design for collaborative video searching interface.


Paper review for ACM SIGIR 2005 (March)
I reviewed papers for SIGIR 2005 - 28th Annual International ACM Conference on Research and Development in Information Retrieval (Brazil, 15-19 August 2005).


Paper review for CIVR 2005 (March)
I reviewed papers for CIVR 2005 - International Conference on Image and Video Retrieval (Singapore, 20-22 July 2005).


Sorin's system: Advanced with query split (March)
February's system of Sorin/I has been further upgraded that considers the inconsistent example objects a user adds to the query panel. In Query-By-Example systems, if the user adds multiple examples that are very different from each other, what happens? The search result will not be as accurate as when the user adds very similar examples, because this will confuse whatever the internal mechanism the system uses to classify the objects. This has been a problem for our TRECVID2004/2003 systems: we don't want the users to add very different images but yet we cannot force them not to because it's a user's legitimate action allowed by the interface. The advanced system is based on Sorin's February system, but it can suggeste to the user the clustering of the query objects the user added. In this way, the user can now focus on only one similar group of example objects, and return to other groups later on. This is in line with the 'ostensive relevance feedback' concept, in that the interface allows the user to persue one line of queries, and after a few interaction, she can come back to where she branched out and persue other lines of queries. This system was submitted to AMR 2005. This time I focused much more on the coherent look-and-feel (a figure beside), and tried to make all the buttons, background, labels and objects as integrated with each other as possible. Also the main feature of the system - splitting the query objects - was emphasised appropriately.


Interface design for Sorin's object-based system (January - February)
The idea of object-based interaction I have been developing was applied to Sorin's relevance feedback system. Because the most important element of this sytem is relevance feedback from the user who indicates, for each object, its low-level features including colour, shape and texture, I focused on visualising this in an easily understandable way. Iterative query formulation occurs by selecting buttons for each feature. The buttons I designed was a little strange(?) widget - each feature has 3 state: positive, neutral and negative. Clicking on a feature will rotate these states. For different state I tried to give a distinctive colour, and Alan's suggestion on position difference was also later taken up (see some examples in the image beside:). However, I do not think this button set goes very well with the rest of the interface - an important aspect of interface is that all elements within in should be contributing to a particular 'theme' resulting in overall coherence and unity. I don't think I achieved this with this system (particularly due to this button set - I designed it in a separate way and just added in)... in the future, more complete design would consider the overall scheme, and any widget like this button set will have to bear some common characteristic with the rest of the interface.


CCTV interface to ICDP Conference (February)
Based on my CCTV interface of December last year, I wrote and submitted a paper to ICDP 2005 focusing on how this interface can be implemented with the use of various object-based techniques we are developing now: object detection and tracking, pedestrian detection, use of infrared superimposed with visible signal to detect human's movement more accurately, etc.


ESSIR 2005 publicity starts (January)
Our Centre is hosting this year's ESSIR (European Summer School in Information Retrieval) here in Dublin in September. As a 'Publicity Chair' of the event, I have designed a website for this... Convenient thing about 'minimalist' design is that I don't need a lot of time to physically design/implement it - once the idea is there, it's easy and simple to develop. Although web design is directly related to my research (thus in fact I think I do have all good excuses for fiddling with web page design and "colour and all"), I wanted this to be quick and effective in terms of me allocating my time into it. Besides, I have many other important things to do. So I quickly came up with a 'theme' of the website, then designed a scheme for exploiting this, by designing background image, line sparator and paragraph starter images. A good design usually uses minimal use of 'administrative debris' such as lines, and instead, uses the actual information itself for aligning, separating, and providing style (that's how I learned anyway). Anyway the result is the ESSIR 2005 website, cheap and quick to produce, but (hopefully) effective and with its unique style. In particular, the round icons of lecturer face photos are used in different places, combining and linking and providing a theme threaded by lecturer faces (similar to the way I used feature icons for video shots in Físchlár-TREC2002 system?). An 1-page flyer I designed later on (shown beside) bears the same style.


my reading    
in 2005    
It's part of my job to keep reading recent books and papers in my field of research to get me informed and to get my head turning in the right direction. Monographs are difficult to read during busy daily work, almost a lunxury to me now, but following is the one thing I managed to do during Easter:

Norman, D. Emotional Design: Why We Love (Or Hate) Everyday Thing. Basic Books, 2004.

This is a kind of "sequal" to his well-known 'Design of Everyday Things' (1988) which I was fascinated reading in 1998. Emphasis on usability was the main theme of the 1988 book, and from that there were many claims from people that 'following his ideas did make the system more usable, but it made my system look terrible, boring, and unattractive. Normal's 2005 book is to address this aspact, that his older book was not trying to degrade the aesthetic aspect of design but to enhance the level of usability up. His new book is interesting in that it divides 3 different levels of design and the usability aspect fits nicely as one of the levels.
  • Visceral level
  • Behavioural level
  • Reflective level
The conventional idea of doing usability testing, iterative refinement based on testing are only addressing the Behavioural level of design - currently well underway and well recognised. However, the other two levels of design, often relying on individual designer's flair and graphic designer's skills, are not well understood especially not as part of user interface design. I was very intrigued by the way Norman combined these aspects with the usability aspects together.

Also, in 2005 I have been doing more focused reading, and some of the representative or useful ones I read are the following. I started reading on CCTV and security related subjects - my ICDP 2005 paper was shaped from a few important papers including:

Lipton, A., Heartwell, C., Haering, N. and Madden, D. Critical asset protection, perimeter monitoring, and threat detection using automated video surveillance. ObjectVideo white paper. 2003.

Lipton, A. ObjectVideo Forensics: activity-based video indexing and retrieval for physical secrurity applications. ObjectVideo white paper. 2003.

Huston, L., Sukthankar, R., Campbell, J. and Pillai, P. Forensic video reconstruction. Proceedings of the ACM 2nd international workshop on Video surveillance & Sensor Networks, 2004, pp 20-28.

Siebel, N. and Maybank, S. The ADVISOR Visual Surveillance System. Applications of Computer Vision '04, 2004, pp 103-111.

Hampapur, A., Brown, L., Connell, J., Haas, N., Lu, M., Merkl, M., Pankanti, S., Senior, A., Shu, C. and Tian, Y. S3-R1: the IBM Smart Surveillance System - Release 1. ETP'04, 2004, pp 59-62.

Working with Sorin on his interactive object-based system, I looked up papers related to relevance feedback, especially on image/video systems - these are some of the papers that helped me design Sorin's object retrieval system:

Lu, Y., Hu, C., Zhu, X., Zhang, H. and Yang, Q. A unified framework for semantics and feature based relevance feedback in image retrieval systems. ACM Multimedia 2000, 2000, pp 31-37.

Ruthven, I. Incorporating aspects of information use into relevance feedback. Journal of Information Retrieval, 1999, pp 1-5.

Worring, M., Nguyen, G., Hollink, L., van Germert, J. and Koelma, D.C. Interactive search using indexing, filtering, browsing, and ranking. Proceedings of TRECVID 2003, Text Retrieval Conference Video Track, 2003.

Haas, M., Rijsdam, J., Thomee, B. and Lew, M. Relevance feedback: perceptual learning and retrieval in bio-computing, photos, and video. Proceedings of the ACM Multimedia Information Retrieval (MIR'04), 2004, pp 151-156.

Ojala, T., Koskela, M., Matinmikko, E., Rautiainen, M., Laaksonen, J. and Oja, E. Task-based user evaluation of content-based image database browsing systems. Proceedings of the International Conference on Image and Video Retrieval (CIVR 2004), 2004.

Heesch, D. and Ruger, S. Three interfaces for content-based access to image collections. Proceedings of the International Conference on Image and Video Retrieval (CIVR 2004), 2004.

Koskela, M., Laaksonen, J. and Oja, E. Use of image subset features in image retrieval with self-organizing maps. Proceedings of the International Conference on Image and Video Retrieval (CIVR 2004), 2004.

From April, I've started working on DiamondTouch: the MERL's tabletop system on which we are developing video search system. Tabletop is an interesting area for research, needing much novel ideas and a lot of scope for detailed study. Social aspect of collaborative interaction is a whole new area, although overlapping the usual single-user interaction design. Interaction design for mobile devices and for tabletop devices is maybe the extreme opposite cases of interesting challenges for HCI work. There are many interesting papers so far I have managed to find and read. In individual system or development level, especially following ones I found useful:

Tandler, P., Prante, T., Muller-Tomfelde, C., Streitz, N. and Steinmetz, R. ConnecTables: dynamic coupling of displays for the flexible creation of shared workspaces. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'01), 2001, pp 11-20.

Streitz, N., Geisler, J., Holmer, T., Konomi, S., Muller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P. and Steinmetz, R. i-LAND: an interactive landscape for creativity and innovation. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'99), 1999, pp 120-127.

Ashdown, M. and Robinson, P. Escritoire: a personal projected display. IEEE Multimedia, 12(1), 2005, pp 34-42.

Patel, D. Habitat: awareness of daily routines and rhythms over a distance using networked furniture. Proceedings of London Communications Symposium (LCS 2003), 2003, pp 145-148.

Matsushita, M., Iida, M. and Ohguro, T. Lumisight Table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. Proceedings of the ACM conference on Computer Supported Cooperative Work (CSCW '04), 2004, pp 274-283.

Patten, J., Ishii, H., Hines, J. and Pangaro, G. Sensetable: a wireless object tracking platform for tangible user interfaces. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'01), 2001, pp 253-260.

Blackwell, A., Stringer, M., Toye, E. and Rode, J. Tangible interface for collaborative information retrieval. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI2004), 2004, pp 1473-1476.

Shoemaker, G. and Inkpen, K. Single display privacyware: augmenting public displays with private information. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'01), 2001, pp 522-529.

Stewart, J., Raybourn, E., Bederson, B. and Druin, A. When two hands are better than one: enhancing collaboration using single display groupware. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'98), 1998, pp 287-288.

Among some of the interesting tabletop applications, DiamondTouch system is most comprehensive: has its hardware sorted, and by developing software toolkit called DiamondSpin, various interesting applications are easily built and here are some of the DiamondTouch related papers I found highly useful:

Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'01), 2001, pp 219-226.

Shen, C., Vernier, F., Forlines, C. and Ringel, M. DiamondSpin: an extensible toolkit for around-the-table interaction. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2004), 2004, pp 167-174.

Shen, C., Lesh, N. and Vernier, F. Personal Digital Historian: Story sharing around the table. Interactions, 2003, pp 15-22.

Shen, C., Lesh, N., Vernier, F., Forlines, C. and Frost, J. Sharing and building digital group histories. Proceedings of the ACM conference on Computer Supported Cooperative Work (CSCW '02), 2002, pp 324-333.

Shen, C., Everitt, K. and Ryall, K. UbiTable: impromptu face-to-face collaboration on horizontal interactive surfaces. Proceedings of 5th International Conference on Ubiquitous Computing (UbiComp 2003), 2003.

Then, some of the more generaliable, more useful, experiments and insights for tabletop interaction design are found - these are the ones that guide the design of new tabletop systems, thus I'm most interested in this type of papers:

Tse, E., Histon, J., Scott, S. and Greenberg, S. Avoiding interference: how people use spatial separation and partitioning in SDG workspaces. Proceedings of the ACM conference on Computer Supported Cooperative Work (CSCW '04), 2004, pp 252-261.

Morris, M., Ryall, K., Shen, C., Forlines, C. and Vernier, F. Beyond "social protocols": multi-user coordination policies for co-located groupware. Proceedings of the ACM conference on Computer Supported Cooperative Work (CSCW '04), 2004, pp 262-265.

Kruger, R., Carpendale, S., Scott, S. and Greenberg, S. How people use orientation on tables: comprehension, coordination and communication. Proceedings of International Conference on Supporting Group Work (Group'03), 2003, pp 369-378.

Scott, S., Grant, K. and Mandryk, R. System guidelines for co-located, collaborative work on a tabletop display. Proceedings of Eurpoean Conference on Computer Supported Cooperative Work (ECSCW'03), 2003.

Scott, S., Sheelagh, Carpendale, M. and Inkpen, K. Territoriality in Collaborative Tabletop Workspaces. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'04), 2004, pp 294-303.

Gutwin, C. and Greenberg, S. Design for individuals, design for groups: tradeoffs between power and workspace awareness. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'98), 1998, pp 207-216.

Gutwin, C. and Greenberg, S. Workspace awareness. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'97) Workshop on Awareness in Collaborative Systems, 1997.

Paek, T., Agrawala, M., Basu, S., Drucker, S. and Kristjansson, T. Towards universal mobile interaction for shared displays. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'04), 2004, pp 266-269.

Some of the above seem to say conflicting ideas (understandable considering the stage of tabletop development today)... I continue to read while working with Colum and Sinéad on developing our own DiamondSpin application for video searching, incorporating the guidelines and other findings from the above papers into our own design.

Singapore University of Technology and Design

Design by Hyowon Lee 2012