Research Projects - Tips and Tricks

Congratulations on getting to the research project stage of your degree. On this page, you will find a variety of tips, tricks, and advice, to help your project run smoothly. This page is especially tailored towards projects that involve, primarily, building tech. If you are doing a different style of project, ask me if I have a page that might be more relevant to you. 

Even though you have 2 semesters for your project, it will fly past very quickly, so it is important to optimise your time and be relatively strict with yourself. 

Getting Started

Project Description: Re-visit the project description. Do you understand the project? Do you understand what you will be doing?

Join Slack: You will be invited to join a Slack channel - that will be our main communication channel, outside of our meetings.

Tools: Prep the tools you will likely need for your project (probably some, or all, of the following: Slack, Unity, Overleaf, Zotero, Arduino, Fusion360). Complete necessary training for those tools. You will likely be 3D printing something during your project - determine which 3D printers are the best for you to use, get access, and complete any training. I strongly recommend using Notion to manage and log your research. Take photos of everything as you go!

Papers: Start reading relevant papers. See Reading Papers below, for advice. 

Reading Papers

Reading papers is an art. Also, reading does not necessarily mean 'reading'.  Look carefully at the below diagram on how to read - note that you don't typically read a paper from start to finish!

Your principal aim, in the first couple of weeks, should be to find 3 key papers that are relevant to your work. These are probably the only papers you should read in full (trying to understand every section).  For these papers, answer the following 4 questions in a short sentence:

How do you find these key papers? Ask me for suggestions and key terms. Search for papers using Google Scholar and the ACM digital library. Quickly judge papers' relevance, based on their titles, abstract, images, and maybe videos. Read the intro of papers that pass some relevance threshold (e.g., 'this is probably relevant, so I'll read the intro to be sure').  Still seem relevant? Then keep reading (discussion, then method, then results). 

And after I've found those papers? Once you have found those key papers (and it is ok if they change over time), you can likely use forward- and back-chaining to find other papers. This means looking at who they cited (back-chaining - look at the references) and who cited them (forward-chaining - the ACM Digital Library and Google Scholar both have buttons for this).  You should also just google the first author (primary author - usually a PhD student, who is probably working on a consistent topic) and the last author (their supervisor - working in the broader space probably).

Ideally, you should have ~3 top papers (read the whole way through), ~10 more really relevant papers (read the intro, discussion, and method), and ~10 quite relevant papers (read the intro and watched the video). Keep in mind, papers can be relevant because of their tech, or their method, or underlying theory, etc. 

How can I tell if a paper is of good quality? This is a big question and the real answer to this is probably 'experience'. In place of experience, however, you can often use where the work was published as a (very) rough proxy. In HCI, we have a few tiers of publications. The exact make up of these tiers depend a bit on who you talk to, but roughly:

Top Tier: ACM CHI, ACM UIST, ACM CSCW
 2nd Tier: PACM IMWUT, IEEE VR, ISMAR,
3rd Tier: ACM ISS, VRST, TEI, DIS, CHI Play, Interact, EICS
 and there are loads others, but primarily concentrate on the venues above.

You certainly don't need to only read papers from HCI though, even if you are doing an HCI-style building project. You might find interesting ideas in a wide range of other areas (e.g., material sciences, psychology, graphics, etc.). For simplicity, I would say at least ~90% of your papers should be from HCI.

Let's think about the rest of your work through the paper you have to write

You have to write a report. This report is quite close to an academic paper. Let's think about the sections of that paper. This will help you think about the work you need to do. 

Introduction

In this section, you need to quickly tell people what your research problem is, what others have done, what you have done, and what you found. There is obviously wiggle room, but your intro will probably follow an outline like that detailed below. Each paragraph should be 3-4 short sentences. You should reference related work in your intro. 

Paragraph 1: 

For tech-building work: <A short snappy hook: either (a) what is the problem, or (b) imagine if we could do this>. Research has <demonstrated something, or built prototypes that enable the concept>. We extend the understanding/opportunities of this space, by <prototyping blah>. Our device, <give it a name> offers <something more than the related work>.

Paragraph 2: what has the related work done in this space?

Paragraph 3: what do you do? (what did you build? what kind of study did you run?)

Paragraph 4: what did you find? what did we learn?

Related Work

In this section, you capture all the work you did when reading papers. Your related work should consist of ~3 sections. Give each section a sub-heading. Each section should be a theme that links a subset of the papers you have read together. (Quite possibly, each section will be centered around one of your key papers, but doesn't have to be). Each section should draw parallels between ~5 papers. These parallels could be methodological, technical, result-driven, etc. Tell us about these parallels! Don't just list papers here. For each paper, where it makes sense, give us a one-line insight into what they did and what they found. At the end of each section, you should write 1-2 lines about how you 'use' this related work (either how it informs your work or how you build on it). 

The first mini-paragraph of your related work section should introduce the 3 sections and tell us how they link together / why those sections make sense.

System Design

How are you approaching your research question? Regardless of the type of paper/report you are writing, it is always good to be able to reference other work that has used a similar methodology.

For tech-building papers, you could say that you are following an 'exploring new interactions through prototyping' or 'engineering HCI' method. Many relevant papers will effectively use such a method. (Note: this is not a formal method name, so no one will have written this in their paper. Calling it something specific is simply to help the other researchers in our group who may have to grade your work). Your 'method' or, more likely 'system design' should achieve two things: (1) describe what you did such that the reader could re-create your system, and (2) justify any decisions you made (e.g., why did you use a specific component, material, or algorithm - where non-obvious, or why did you build a certain type of thing). Use a lot of figures in this section! It is good practice to link to a repo where your code or design files can be found. 

Important: There is often an expectation that your work should involve a user evaluation. You do not need to do one, however, as engineering something that someone else could then study is sufficient. If that is your intention (that somebody else studies the system later), then you should clearly state that somewhere in your work. I would suggest including something like this towards the end of your intro: 'We contribute a system design that <very briefly state what it does>. Conducting a user evaluation of this system is outside of our scope and remains future work.' 

Evaluation / Method:

Your work will need some style of evaluation. This may be a user evaluation (you'll need ethics) or it may be a technical evaluation. 

For user evaluations, you should also ideally closely follow a known method. This will probably mean starting with: We conducted a study to understand <blah>. We follow a method similar to <blah> et al. [reference]. After that, you will explain your setup (what tech, what was the space like, etc.), who participants were (average age, how did you recruit them, did you have any special recruitment criteria), and what you asked them to do. Again, it is important that your reader could accurately reproduce your study, so be detailed and specific. 

For technical evaluations, you want to be able to tell the reader that what you built was good. Typically, we might do this by providing measurements of different parts of its functionality. You might measure: how many degrees of freedom it has, what range of motion it offers, how heavy it is, how accurate it is, how repeatably accurate it is, how robust it is, how long its battery lasts, etc. The things that you measure should be relevant to your contribution. So, for example, if you are building a haptic controller with higher resolution than something else, then measure / evaluate how high that haptic resolution is (e.g., it can do this range of movement, at this many degrees per second, etc.). If at all possible, contrast that with the competition. For a technical evaluation, you still need to detail how you conduct that evaluation, so that it is repeatable and people can judge how good/accurate your evaluation is. 

Results / Analysis: 

Next up, you need to present the results that you gathered and walk us through how you interpret those results. 

First, present the Results. This should be dispassionate and without any interpretation - just present the data. For quantitative data, you should present the results of any statistics. So, what was higher than what, what was significantly different to what, etc. Use figures to help the reader interpret. For qualitative data, present your results after your thematic analysis (so present the data according to the themes). Just tell the reader what you saw participants do and give us quotes about what they said. 

Next, present the Analysis. (Note: sometimes this section is skipped and its contents are just rolled into the Discussion section). Here, you can tell the reader what you interpret from the results. So, in what ways was your technique better or worse than the competition? Its ok if your system was worse, just try and explain why!

Discussion

Very brief recap: What did you do? Why did you do it? What did you learn? 

What might this mean for the future of HCI? If someone comes along to continue this work, what should they do / not do? 

Refer back to the literature in this section, re-highlighting what was known before and thinking about what your work really adds to that.

You can afford to be a bit speculative in this section (especially in a thesis) - what are new unknowns here, that are now interesting? Do your results make you question any existing results?

Conclusion

Brief recap summary of the whole paper. 1 mid-length paragraph max!

Notes on Writing

I do not doubt that you will use chatGPT to help with your writing. That's ok with me (as long as you are honest and make sure you record all your prompts, etc.), but I expect you to put in work to do it well. Lazy prompt engineering results in writing that is too passive and too formal for HCI (its not enough to simply say 'make it sound like a scientific journal' in your prompt). You need to tailor specifically to HCI, where writing is more active, a little less formal, and sometimes tends towards journalistic. In short, there should be an essence of 'humanity' in your writing, which GPT would typically remove. 

The best way to write well is to find a relevant paper that you like and write your paper to sound as much like that paper as you can! 

If I ask for a draft of something (like an intro, for example) and you put in minimal effort, take something dumped out of GPT and just give it to me, I will be angry. If you work hard for me, I will work hard for you. If you are lazy for me, I'll let you fail...

Your timeline

'There is much to do and less time to do it in'. 

I would encourage you to manage your time very carefully. Set yourself deadlines and stick to them. You shouldn't expect things to work first time, especially with prototyping, so build in time for iteration. Also, plan for writing to take longer than you think!

The university believes you should be spending ~10 hours / week on this research project. I would encourage you to try and stick to that throughout - this will keep you on track.

The precise breakdown of your time is up to you, but I would suggest something like the following:

Weeks 1 - 2: Finalise your supervisor. Start doing some reading. Start getting more specific about your exact topic area. Where do you think there is a gap for you to contribute?

Week 3: Start sketching ideas. At this stage, sketch as many different ideas as possible. For our week 3 supervision meeting, I'd like to see ~15 ideas. These can be entirely distinct / unique, different ways of achieving the same thing, refinements of other people's ideas, or whatever. They don't need to be hugely detailed. No sketch should really take more than ~1 minute to actually get onto paper. And don't spend too much time thinking, either.  'The best way to have good ideas is to have lots of ideas'.

At the same time as this, get yourself up to speed with any new tools or software that you will likely need. This might involve doing Unity, Arduino, or Fusion360 demos, for example. 

Week 4: Keep sketching. Start refining 'good' ideas (e.g., adding more detail, thinking about some specifics such as how joints will work, where motors will go, etc.). We still want multiple options at this stage, so maybe keep 3 ideas alive for now. Keep learning relevant skills on the side. 

Week 5 - 6: Start digitising your 'best' idea into Fusion360. We want to start moving towards our first physical prototype. This will still probably take a couple of weeks. Don't worry about things being perfect. It's really useful to have something to play with, try out, and look at, even if its a bit rubbish. Print often, because it will take some time to learn about tolerances, etc. 

~Week 7: Let's sit down and look at your prototype. What works? What doesn't? Let's think about how to refine it. 

Weeks 8 - 12: Work on version 2 and 3 (and, maybe, 4, and 5). Try new things. Keep refining. Take notes on what we are learning as we go. As we get towards a stable physical prototype, you will increasingly turn your attention to the digital experience part (e.g., adding smarts to Arduino, building Unity apps, etc.). 

End of First Semester Target: a somewhat workable prototype device. 

At this point, the specifics of your project will likely take you in different directions. In general, we will likely (a) refine the whole prototype experience, (b) plan our evaluation (whether user study or technical), (c) conduct our evaluation, and (d) write up.