ChatGPT’s potential for human-like communication is noteworthy, but the mental health implications of integrating a real human identity remain understudied. This research focuses on introducing an AvatarGPT, a human-like avatar, to the ChatGPT interface to delve into these effects. A between-subject study (N=10) was conducted to investigate users’ responses, evaluate, and compare the effectiveness of AvatarGPT and ChatGPT before and after a conversation using the UCLA Loneliness Scale. Results show that neither using an avatar (p≈0.39) nor conversing without the avatar (p≈0.11) significantly improved loneliness scores. Additionally, using the avatar did not enhance willingness to speak, as measured by increased word counts, or significantly reducing loneliness score in percentage (p≈0.59). We believe that with a larger participant pool and a longer experimental period, we would be able to observe a more significant increase in emotional change.

Introduction

In recent years, exciting technological advances and research in robotics have led to a wide range of applications in many areas. One kind of robotics called Socially Assistive Robots(SAR) is widely used in health care and emotional support areas for it has the ability to provide emotional, cognitive and social support. These robots have been used to combat cognitive decline and depression through various methods such as making jokes and dancing. However, they provide emotional support only through basic reflective listening techniques and simple interactions, such as repeating or paraphrasing people’s words and their conversation patterns could use a lot of improvement.

Meanwhile, advancements in natural language processing and AI have given rise to innovative tools that offer a new dimension of human-computer interaction. ChatGPT, a language model developed by OpenAI, possesses the ability to understand and generate natural language, enabling smooth conversational interactions with users. Its natural language processing capabilities present immense potential for applications in emotional support and mental health.

Integrating an avatar interface with ChatGPT is primarily motivated by the limitations of text-only input in traditional chatbot interactions. By incorporating an avatar-based interactive interface, it is possible to enhance user engagement and enrich the overall experience. This approach is expected to foster greater initiative in user interactions with the chatbot, making the exchanges more intuitive and personable and improving user interaction and satisfaction.

The integration of LLMs like ChatGPT represents a further leap in mental health care. LLMs offer more natural, deeper conversational interactions, crucial for personalized emotional support and therapy. Incorporating LLMs into SAR with virtual avatars enriches the user experience, providing a sense of real social interaction and deeper emotional resonance and opening new possibilities for personalized and emotional support.

Thus, we implemented a new chat interface called AvatarGPT, ChatGPT with a vivid avatar interface to ’speak out’ the texts answered by ChatGPT, to provide mental health care. We then conducted a between-subject experiment with 10 participants in which some participants were using AvatarGPT (the treatment group) and some were given a basic ChatGPT (the control group). We formulated our research question as follows: Will the presence of a human-like avatar in the interface of ChatGPT increase the feeling of human connection during a conversation and, consequently, help reduce the symptoms of depression, especially loneliness?

To address this research question, we considered the following hypotheses:

H1:

Conversing with ChatGPT with an avatar would reduce user loneliness

H2:

Use avatar would increase speaking willingness

H3:

Conversations with more words would indicate greater reduced loneliness

Contribution

Surprisingly, we found that neither ChatGPT nor AvatarGPT significantly contributes to people’s mental health in terms of reducing loneliness. In addition, we didn’t observe an increased level of willingness to speak or a more significant reduction in loneliness percentage across the two conditions.

Related work

In the field of mental health care, the application of Socially Assistive Robots (SAR) is increasingly being emphasized, especially with the development of AI. Mental health issues pose a significant burden on individuals and society. In the US, about 25% of the population meets the criteria for a diagnosable psychiatric condition each year. Against this backdrop, SAR demonstrates tremendous potential. These robots have already assumed various roles in mental healthcare, including companions, coaches, and play partners.

For example, a robot called Paro has been used as a companion providing emotional support and reducing loneliness, akin to trained therapy animals. Also, simple robots with basic conversational abilities have proven beneficial for otherwise healthy older adults facing social isolation and loneliness.

AI now also become a valuable tool in this field, aiding professionals and enhancing patient care. Some AI systems, like Horyzons, provide accessible social therapy for youths with psychosis, allowing them to share and express emotions in a safe, moderated online environment. This platform reduces the intimidation associated with traditional group therapy by facilitating anonymous interaction. Also, Online Mental Health Communities use AI-mediated communication to better form relationships and communicate between people.

Method

We conducted a between-subjects experiment to investigate the influence of interactions with ChatGPT and AvatarGPT on participants’ self-reported loneliness levels. Participants, recruited from a campus setting, were randomly assigned to one of two groups. Each group engaged in interactive tasks with either ChatGPT or AvatarGPT(modified to include a speaking humanoid avatar integrated into the ChatGPT interface). The experiment included a pre-survey to establish baseline loneliness, 2 interactive tasks for each group, and a post-survey for reassessment. Our study aims to find out how these interactions impact participants’ emotional states and feelings of loneliness.

Pre-Survey: Establish Baseline Level of Loneliness

Prior to the start of the experiment, participants were asked to complete the modifed UCLA Loneliness Scale survey independently, without any integration with ChatGPT. By comparing the pre-survey responses with subsequent post-survey responses, we can effectively measure any changes in participants’ feelings of loneliness resulting from their interactions with AvatarGPT.

Task 1: Question-Answering Task

Participants in this task will engage in a conversation with either ChatGPT or AvatarGPT, during which they will respond to questions related to their personal life, hobbies, or daily challenges. The primary goal is to encourage participants to share their feelings and thoughts openly, as well as to prompt them to pose additional questions. To initiate the task, ChatGPT or AvatarGPT is given the following prompt: “I need you to act like the interviewer, so speak like a human. I need you to ask me questions about life, hobbies, or daily challenges. Engage in a conversation with me. I need you to talk to me and encourage me to share my feelings and pose additional questions. Ask specific questions. Start off the conversation with ’Thank you for coming!’”

The participants are instructed to chat for a duration of 5 minutes. Importantly, during this time, the experimenter should not intervene. After the 5-minute conversation, the task will conclude, and participants will proceed to Task 2.

Task 2: Scenario Generation Task

In this task, participants will engage in collaborative storytelling with either ChatGPT or AvatarGPT, focusing on themes of friendship and support. The objective is to jointly create a story plot that embodies these themes, while also integrating personal experiences and emotional elements to add authenticity and depth to the narrative. Participants are encouraged to work together to construct a story that flows seamlessly and feels interconnected. To initiate the task, participants are instructed as follows: “The goal of this task is to collaboratively create a story plot that revolves around themes of friendship and support in the next 5 minutes. We encourage you to infuse your personal experiences and emotions into the narrative to make it authentic and meaningful. Remember, this is a collaborative effort. Feel free to build on each other’s ideas, creating a narrative that flows seamlessly. Collaboration is key to making our story rich and interconnected. Let’s start with a simple prompt to kick things off. Imagine a setting or a situation where the characters need friendship and support. It could be anything – a challenging moment, a celebration, or even a quiet, reflective scene. Build upon this prompt in your contributions.” After the 5-minute storytelling session concludes, the conversation will stop, and participants will proceed to the post-survey.

Post-Survey: Reassessment of Participant Responses

Following completion of the experimental tasks, participants will be asked to take the same survey that they completed as the pre-survey. The purpose of this post-survey is to assess whether there have been any changes in participant responses or perceptions after engaging in the interactive tasks with ChatGPT or AvatarGPT. By comparing pre-survey and post-survey responses, we aim to examine the potential impact of these interactions on participants’ feelings, experiences, and attitudes. This comparison will enable us to gain insights into the effectiveness of the interactions and their potential influence on various aspects of participants’ well-being and emotional state.

Participants and recruitment

We recruit 10 people via interactions with students on campus. All of them self-identified as 1) at least 18 years old.

Experiment description

In our experiment, we introduced the AvatarGPT by adding an avatar to the right bottom corner of the website ChatGPT interface in the browser window, as shown in 1. The modified interface will be represented to participants in Task 1 and Task 2.

Specifically, we employed a JavaScript program through Tampermonkey to integrate a humanoid face into the corner of the ChatGPT interface. This humanoid face was designed to simulate speaking by moving its mouth synchronously with the textual responses generated by ChatGPT. The primary objective was to create an association in the user’s mind, where they would attribute the words produced by ChatGPT as being spoken by the avatar.

A sample running environment for AvatarGPT

Data collection

In our study, we used a slightly modified UCLA Loneliness Scale to measure participants’ loneliness levels [bit.ly/cse216survey1]. The UCLA Loneliness scale is a well-established questionnaire for assessing loneliness that employs a frequency-based response format. However, we adapted it by retaining the same questions but using an agreeance-based response scale. Participants were required to indicated their agreement levels with 20 negatively phrased statements. This allowed us to gauge participants’ current loneliness within the context of our short-term experiment. Lower scores indicate a more optimistic emotional state, while higher scores reflect a more pessimistic one.

In more detail regarding the evaluation and calculation of scores, the participant’s responses are scored on a scale from 1 to 5 for each question. A score of 1 represents “Strongly Disagree,” indicating that the participant completely disagrees with the negative statement, suggesting a more optimistic emotional state. Conversely, a score of 5 represents “Strongly Agree,” indicating a high level of agreement with the negative statement, which may suggest a more pessimistic emotional state. Once we finish converting all answers from categories to scores, we simply use the average overall responses as the final representation of one’s loneliness scale.

Result

10 people are randomly assigned to two groups, either AvatarGPT group or ChatGPT group. Among 10 participants, 6 (2 Male, 3 Female, 1 prefer not to say) were assigned to ChatGPT group. The age is between 19-25 (M=21.3, Median=21). 4 participants (2 Male, 1 Female, 1 prefer not to say) were assigned to AvatarGPT group. The age is in between 21-24 (M=21.7, Median=21.5).

We collected the score based on 3.2 and analyzed the effect of interactions on participant loneliness for each group by addressing each hypothesis one by one.

H1: Conversing with ChatGPT with an avatar would reduce user loneliness

We calculated the score based on the method mentioned in 3.2, representing the final result as shown in 2 for both groups. We then conducted a Student paired t-test to assess the impact of the interactions for each condition. In ChtGPT group, where participants engaged with the original interface of ChatGPT, we did not observe a significant difference in scores before and after the chat (t≈2.2, p≈0.11). Similarly, in AvatarGPT group, where participants interacted with the avatar version of ChatGPT, we also did not find a significant difference in scores before and after the chat (t≈2.4, p≈0.39). Lower scores indicate lower levels of measured loneliness.

A Student paired t-test shows neither ChtGPT nor AvatarGPT has a significant contribution to reducing an individual’s loneliness score.

H2: Use avatar would increase speaking willingness

From the recorded conversations, we analyzed both the word counts from the subjects and those generated by ChatGPT or AvatarGPT. The result is shown in 3 for each condition.

We conducted a Student paired t-test to analyze the relationship between total word counts and avatar usage. Our results indicated no significant difference between the usage of the avatar and participants’ willingness to type (t$\approx$2.4, p$\approx$0.83).

A Student paired t-test shows using an avatar would not increase the willingness to speak significantly.

H3: Conversations with more words would indicate greater reduced loneliness

To quantify the changes in feelings of loneliness, we assess these variations using the percentage change in scores from the Pre-Survey. This method involves comparing the scores of the same participant across two separate surveys and calculating the difference between these scores. Specifically, we record the scores of a participant from two different instances of the survey. Then, we calculate the difference between these two scores. This difference is then divided by the score from the Pre-Survey, resulting in a value that represents the change in feelings of loneliness. This value reflects the extent of change in loneliness between the two tests, essentially representing the proportion of the change in scores relative to the score of the first test. The result for each condition is shown in 4.

We conducted an ANOVA test to examine the relationship between word counts and the change in scores expressed as a percentage. Our analysis revealed no significant difference between these variables (F(1, 8) $\approx$0.31, p$\approx$0.59, $\eta^2\approx$0.04). It’s worth noting that a negative score indicates that participants reported heightened feelings of loneliness after the conversation.

An ANOVA test shows using an avatar would not reduce loneliness in terms of the ratio of the score difference.

Limitation

One notable limitation of our study is the lack of validation for the modified version of the UCLA Loneliness Scale employed in this research. While the UCLA Loneliness Scale is well-established measuring loneliness, our adaptation was designed to assess transient or short-term loneliness as a mood rather than a traditional emotional state. Given that there was no widely recognized scale available specifically tailored to the measurement of transient loneliness in the context of our study, this adaptation was a necessary step. However, this modification raises questions about the scale’s validity and whether it accurately captures the targeted construct. future research should consider conducting validation studies to assess the reliability of our adapted scale for measuring short-term loneliness.

Another limitation of our study is the specific avatar chosen for the interactions with participants. We selected an arbitrary humanoid woman avatar presented in black and white, which was aligned with the color scheme of ChatGPT. Individual perceptions of avatars can vary widely, and some participants reported the chosen avatar to be “scary” or “distracting”. A more systematic approach to avatar selection should be considered.

Future work

Exploring the impact of personalized avatar designs and behaviors on user engagement is important as customization options for avatars could enhance interactions. Investigating the long-term effects of both ChatGPT and AvatarGPT on user well-being and attachment to avatars. Comparative studies with other mental health support tools can provide insights into AvatarGPT’s unique advantages and limitations in different contexts, contributing to more effective and responsible AI-driven mental health care systems.

Conclusion

Even though both robotics and large language models are trying to simulate human behavior, our current research result reveals the distance between them and real humans. In this research, our results show that neither using an avatar nor conversing without the avatar significantly improved loneliness scores. Additionally, using the avatar did not enhance willingness to speak, or significantly reduce loneliness score in percentage. We believe that with a larger participant pool and a longer experimental period, we would be able to observe a more significant increase in emotional change.

The original PDF file

In this project, we

• Designed the core logic in pure functional language
• Wrote the GUI in pure functional language
• Designed a user-friendly interface
• Did property tests to catch logical errors

Overview

This project implements Conway’s Game of Life by using Haskell, which is great for modeling complex mathematical concepts with the design of Haskell. The Game of Life, a zero-player game, evolves from an initial state, showing emergent behavior based on some rules and status (Birth, Survival, and Death). Haskell’s strong typing and pure functionality align perfectly with this game.

Game Description

The Game of Life occurs on an infinite grid of cells. The status of each cell will be changed based on neighboring cells following those rules:

• Birth: A dead cell with three live neighbors becomes alive.
• Survival: A live cell with two or three live neighbors stays alive.
• Death:
  • Overpopulation: A live cell with over three live neighbors dies.
  • Loneliness: A live cell with under two live neighbors dies.

implementation

Haskell’s fits for expressing the game’s rules. Key aspects of our implementation include:

1. Functional Grid Representation: Using list comprehensions and higher-order functions for the grid.
2. State Transformation: Pure functions for state transitions between generations.
3. Interaction Functionality: Implementing pause/continue in the game, boosting the interaction between users and the game with the help of the brick library.
4. Unit Testing: Using QuickCheck to test each functionality, and game logic.
5. Managing Dependencies: Making use of tools like Cabal or Stack to manage dependencies.

Project Goals

1. Mathematical Fidelity: Ensure full implementation of the game following Conway’s rules.
2. User Interactions: Interacting with a user-friendly interface by key presses for pausing/continuing the game or quitting using the brick library.
3. Using existing libraries of the hackage ecosystem: Utilizing existing libraries like containers and vector.

This project aims to create a Conway’s Game of Life simulation in Haskell, as well as representing a string connection between a mathematical model and functional programming.

Source acknowledgment

1. As required by the course website, we are using the Brick library to implement the GUI interface. We did look at the official guidelines from https://github.com/jtdaugherty/brick to implement any components relevant to GUI because the Brick library has never been taught in class.
2. We use Haskell’s Hoodle to explore and employ functions for implementing our project because not all Haskell components were covered during class.
3. We use QuickCheck library to accomplish the unit testing requirments as required by the course website.

Code

The following shows codes for the front-end interface based on the brick library. In more detail, the GUI contains clickable buttons, a user-friendly setting interface, and a playable game with everything being programmed using state in Haskell:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188

clickable{-# LANGUAGE CPP #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TemplateHaskell #-}
module Main (main) where

import qualified Life       as L
import qualified PresetGrid as P
import Control.Concurrent (threadDelay)
import qualified Data.Text as T

import Lens.Micro ((^.), (%~), set)
import Lens.Micro.TH (makeLenses)
import Lens.Micro.Mtl (use, (.=), (%=))
import Control.Monad (void)
import Control.Monad.Trans (liftIO)
#if !(MIN_VERSION_base(4,11,0))
import Data.Monoid ((<>))
#endif
import qualified Graphics.Vty as V

import qualified Brick.Types as T
import Brick.AttrMap
import Brick.Util
import Brick.Types (Widget, ViewportType(Vertical), EventM
  , BrickEvent(..) 
  )
import qualified Brick.Widgets.Center as C
import qualified Brick.Widgets.Border as B
import qualified Brick.Widgets.Border.Style as BS
import Brick.Main
  ( App(..), neverShowCursor, defaultMain
  , suspendAndResume, halt, getVtyHandle, showFirstCursor, vScrollBy, viewportScroll
  )
import Brick.Widgets.Core
import qualified Data.Array as A
import Life (GridState(..))

data Name = Block | Beehive | Loaf | Boat | Tub | Blinker | Toad | Beacon | Reset | Next | Quit | Cell { _idx :: L.GridIndex , _cst :: L.CellState}
          deriving (Show, Ord, Eq)

data St =
    St { _clicked :: [T.Extent Name]
       , _lastReportedClick :: Maybe (Name, T.Location)
       , _state :: L.GridState
       , _step :: Int 
       , _msg :: String
       }

makeLenses ''St

drawUi :: St -> [Widget Name]
drawUi st =
  [vBox [ 
          B.hBorderWithLabel (str "haskell game of life")
         , vBox [ buttonLayer st
             <+> B.vBorder
             <+> C.vCenter (padAll 3 $ hLimit 10 $ vBox [ strWrap ("Click to modify state\n\nCurrent:") , padTop (Pad 1) (gridLayer st L.gridRows L.gridCols)])
                  , str "(Press Esc to quit or n for the next state)" ]]]

gridLayer :: St -> Int -> Int -> Widget Name
gridLayer st rs cs = vBox (map (\r -> hBox (map (\c -> mc (r, c) (_state st)) [0..cs-1])) [0..rs-1])
    where
        mkCell cell = 
            clickable cell $
            --    padTopBottom 1 $
            --    padLeftRight (if wasClicked then 2 else 3) $
            str (if _cst cell == L.Alive then "o" else ".")
        mc idx (GridState g)  = mkCell (Cell idx (g A.! idx)) --str (case g A.! idx of
            --L.Alive -> "o"
            --L.Dead  -> ".") 
        -- cellChar L.Alive = "o"
        -- cellChar L.Dead  = "."

buttonLayer :: St -> Widget Name
buttonLayer st =
    vBox [(padLeft (Pad 1) $ str ("Current steps:" <> show (_step st))) , B.hBorder,
    (padLeft (Pad 1) $  str "Select profile (always live):") 
    ,(hBox $ padAll 1 <$> buttons), 
    vBox [ (padLeft (Pad 1) $  str "Select profile (oscillators):") 
    ,(hBox $ padAll 1 <$> buttons1), (padLeft (Pad 1) $  str "Controls:") 
    ,(hBox $ padAll 1 <$> buttons2)]]
    where
        buttons = mkButton <$> buttonData
        buttonData = [ (Block, "Block", attrName "Block")
                     , (Beehive, "Beehive", attrName "Beehive")
                     , (Loaf, "Loaf", attrName "Loaf")
                     , (Boat, "Boat", attrName "Boat")
                     , (Tub, "Tub", attrName "Tub")
                     ]
        buttons1 = mkButton <$> buttonData1
        buttonData1 = [ (Blinker, "Blinker", attrName "Blinker")
                     , (Toad, "Toad", attrName "Toad")
                     , (Beacon, "Beacon", attrName "Beacon")
                     ]
        buttons2 = mkButton <$> buttonData2
        buttonData2 = [ (Reset, "Reset", attrName "Reset")
                     , (Next, "Next", attrName "Next")
                     , (Quit, "Quit", attrName "Quit")
                     ]
        mkButton (name, label, attr) =
            let wasClicked = (fst <$> st^.lastReportedClick) == Just name
            in clickable name $
               withDefAttr attr $
               B.border $
            --    padTopBottom 1 $
               padLeftRight (if wasClicked then 2 else 3) $
               str (if wasClicked then "<" <> label <> ">" else label)

appEvent :: T.BrickEvent Name e -> T.EventM Name St ()
appEvent ev@(T.MouseDown n _ _ loc) = do
        lastReportedClick .= Just (n, loc)
        case n of
            Block -> do 
                state .= (P.strToGrid P.block)
                step .= 0
            Beehive -> do 
                state .= (P.strToGrid P.beehive)
                step .= 0
            Loaf -> do 
                state .= (P.strToGrid P.loaf)
                step .= 0
            Boat -> do 
                state .= (P.strToGrid P.boat)
                step .= 0
            Tub -> do 
                state .= (P.strToGrid P.tub)
                step .= 0
            Blinker -> do 
                state .= (P.strToGrid P.blinker)
                step .= 0
            Toad -> do 
                state .= (P.strToGrid P.toad)
                step .= 0
            Beacon -> do 
                state .= (P.strToGrid P.beacon)
                step .= 0
            Reset -> do 
                state .= P.deadGrid
                step .= 0
            Next -> do 
                state %= L.evolution
                step %= (+ 1)
            Quit -> do 
                halt
            Cell idx cst -> state %= (\s -> L.toggleState s idx)
            
            
appEvent (T.VtyEvent (V.EvKey (V.KChar 'n') [])) = do
    state %= L.evolution
    step %= (+ 1)
appEvent (T.MouseUp {}) =
    lastReportedClick .= Nothing
appEvent (T.VtyEvent (V.EvMouseUp {})) =
    lastReportedClick .= Nothing
appEvent (T.VtyEvent (V.EvKey V.KEsc [])) =
    halt
appEvent ev = 
    return ()

aMap :: AttrMap
aMap = attrMap V.defAttr
    [ (attrName "Block",   V.white `on` V.green)
    , (attrName "Beehive",   V.white `on` V.green)
    , (attrName "Loaf",   V.white `on` V.green)
    , (attrName "Boat",   V.white `on` V.green)
    , (attrName "Tub",   V.white `on` V.green)
    , (attrName "Blinker",   V.white `on` V.blue)
    , (attrName "Toad",   V.white `on` V.blue)
    , (attrName "Beacon",   V.white `on` V.blue)
    , (attrName "Reset",   V.white `on` V.cyan)
    , (attrName "Next",   V.white `on` V.cyan)
    , (attrName "Quit",   V.white `on` V.cyan)
    ]

app :: App St e Name
app =
    App { appDraw = drawUi
          , appStartEvent = do
              vty <- getVtyHandle
              liftIO $ V.setMode (V.outputIface vty) V.Mouse True
          , appHandleEvent = appEvent
          , appAttrMap = const aMap
          , appChooseCursor = showFirstCursor
          }

main :: IO ()
main = do
    void $ defaultMain app $ St [] Nothing P.deadGrid 0 ""

Shaokang Jiang and Michael Coblenz. An Analysis of the Costs and Benefits of Autocomplete in IDEs. In review, ACM International Symposium on Foundations of Software Engineering (FSE 2024)

Key points:

• Worked with Michael Coblenz on the usability analysis of autocomplete.
• Designed and executed an experiment with 32 participants using an eye tracker to evaluate the costs and benefits of IDE-based autocomplete features to programmers who use an unfamiliar API; analyzed data using JMP; and wrote a paper for the study.
• Found that participants who used autocomplete learned more about the API while spending less time reading the documentation; found autocomplete did not significantly reduce the number of keystrokes required to finish tasks.
• Acquired fundamental skills in conducting empirical research studies and learned various methods for handling and understanding the implications of eye-tracking data.
• Successfully designed an experiment with well-formed hypotheses and a comprehensive knowledge test suite to accurately capture the benefits and costs of autocomplete. Additionally, designed procedures and produced scripts to analyze data from various sources, including a substantial amount of data from a consumer-level eye tracker.

In more detail:

Upon starting my master’s program, I took a usability analysis of programming languages course. Realizing Python type hints could be useful in checking types, and providing more context to IDE for generating useful information, all these properties could be helpful for programmers to write better code. The reduced number of typing errors could potentially assist competitive programmers more by minimizing time spent on fixing typing errors. I investigated the impact of Python type hints feature on the performance of competitive programmers (N=5). Our findings revealed that Python type hints did not significantly enhance their task completion performance (𝑝 ≈ 0.51, 𝑑 ≈ 0.81). Survey results indicated that programmers tended to dislike type hints and autocomplete suggestions, citing them as annoying.

Together with my previous teaching experiences, where I observed programmers grappling with autocomplete suggestions and observed eyes and mouse drifting during active autocomplete use, I aimed to understand the genuine benefits and costs of autocomplete. Considering that autocomplete could potentially be harmful to learning due to the ease of access to knowledge, we conducted an experiment (N=28) to measure the learning impact of autocomplete when programmers face new materials and gauge the productivity effect during its usage. The results are expected to benefit all programmers using modern IDEs, with potential applicability to the general public relying on browser autocomplete.

During the experiment, I utilized an eye tracker to detect users’ gaze, aiming to understand the specific parts and autocomplete options that programmers focused on. I developed a novel monitoring platform based on VSCode, integrating eye-tracking at the source code level. Additionally, I designed a knowledge test suite to measure new knowledge acquisition after tasks and introduced a post-study survey for capturing individuals’ opinions.

I found the main benefit of autocomplete speeding up information acquisition by significantly reducing the time spent reading documentation (𝑝 < 0.01, 𝑑 = 1.76). It also significantly increases the knowledge testing score (𝑝 < 0.01, 𝑑 = 1.1). An insignificant result in the total number of keystrokes across experimental conditions (𝑝 ≈ 0.65, 𝑑 = 0.18) helps us assert that autocomplete does not reduce typing time, thus not making programmers more productive. From designing to executing this study, I learned a lot, including ways of determining the area of interest in eye-tracking studies, methods of designing participant-friendly tasks that align with the project’s goals, quantitative analysis techniques, and methods for hypothesis testing.

I faced numerous challenges during my research project. Integrating an eye-tracker allowed me to delve deeper into the usefulness of autocomplete and the involved cognitive processes, but it also presented challenges. Given the limitations of the previous monitoring platform, including a restricted area of interest, a less popular platform, high latency, and artificial coding restrictions, I developed a new monitoring platform based on the more popular VSCode, integrating eye-tracking at the source code level. I tested various eye trackers, wrote code to evaluate fixation algorithms, and consulted with I-trace experts for optimization insights.

Experiment design posed another significant challenge, requiring the creation of a robust, controllable, and effective experiment that could reveal the benefits or drawbacks of autocomplete. After proposing various experiment versions and conducting pilot experiments, I selected a design utilizing a task suite based on common-sense knowledge and a modern API library. This design included a knowledge test suite, an eye tracker, and a post-study survey. The successful execution of the entire experiment suggests that I effectively overcame these challenges, showcasing my ability to be both persistent and resourceful in graduate school.

Fixation

IDT

One of the most common algorithms for detecting fixations is the Identification by Dispersion-Threshold (IDT) algorithm. According to Salvucci and Goldberg (2000), it is based on the data reduction algorithm by Widdel (1984). The IDT algorithm works with x- and y data, and two fixed thresholds: the maximum fixation dispersion threshold and the minimum fixation duration threshold. To be a fixation, data samples constituting at least enough time to fulfill the duration threshold has to be within a spatial area not exceeding the dispersion threshold. The samples fulfilling these criteria are marked as belonging to a fixation. One particular detail of this specific implementation is that it is part of a package that also merges short nearby fixations, and also paired with a saccade detector (Komogortsev, Gobert, Jayarathna, Koh, & Gowda, 2010).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

// possible time error of using this: 20ms

export class Point {
    curLine = -41;
    /**
     * 
     * @param {Number} x 
     * @param {Number} y 
     * @param {Number} time 
     * @param {Number} curBox y axis location of autocompletion box associate with this points
     */
    constructor(x, y, time, curBox) {
        this.x = x;
        this.y = y;
        this.time = time;
        this.curBox = curBox;
    }

    toString() {
        return "(" + this.x + "," + this.y + "," + this.time + "," + this.curBox + ")";
    }
}

export class Fixation {
    points = [];
    x = 0;
    y = 0;
    time = 0;
    duration = 0;
    line = -1;
    lineAns = -40;
}

class IDT {
    constructor(duration, dispersion) {
        this.duration = duration;
        this.dispersion = dispersion;
    }

    maxBy = (comparator, array) =>
        array.reduce((acc, val) => comparator(acc, val) > 0 ? acc : val);
    minBy = (comparator, array) =>
        array.reduce((acc, val) => comparator(acc, val) < 0 ? acc : val);

    compareX = (a, b) => a.x - b.x;
    compareY = (a, b) => a.y - b.y;

    computeGazeDifference(points) {
        if (points.length == 0) return 0;
        let xmax = this.maxBy(this.compareX, points).x;
        let xmin = this.minBy(this.compareX, points).x;
        let ymax = this.maxBy(this.compareY, points).y;
        let ymin = this.minBy(this.compareY, points).y;
        // console.log(xmax, xmin, ymax, ymin);
        return ((xmax - xmin) + (ymax - ymin));
    }// Not a real dispersion: why not using square root: Pythagorean theorem a^2 + b^2 = c^2 to measure the difference: Most of available code online is using this

    // guessing original step 1 is check whether the point is valid

    calculateTime(points) {
        return points[points.length - 1].time - points[0].time;
    }

    generateFixation(points) {
        let fixation = [];// array of Fixation
        let window = [];//array of points for each run
        let i = 0;
        let tmpDuration = 0;
        while (tmpDuration < this.duration && i < points.length) {
            window.push(points[i]);
            tmpDuration += points[i].time;
            i++;
        }

        while (i < points.length) {
            if ((this.computeGazeDifference(window) <= this.dispersion) && (this.calculateTime(window) >= this.duration)) {
                while (this.computeGazeDifference(window) <= this.dispersion) {
                    if (i < points.length - 1) {
                        window.push(points[i]);
                        ++i;
                    }
                    else { break; }
                }
                fixation.push(this.computeEstimate(window));
                window = [];
                window.push(points[i]);
                i++;
            } else if (this.calculateTime(window) < this.duration) {
                window.push(points[i]);
                i++;
            }
            else {
                window.shift();
                if (this.calculateTime(window) < this.duration) {
                    window.push(points[i]);
                    i++;
                }
            }
        }

        return fixation;
    }

    computeEstimate(points) {
        let fixation = new Fixation();
        let x = 0, y = 0;
        for (let i of points) {
            x += i.x;
            y += i.y;
            fixation.points.push(i);
        }
        fixation.x = (x / fixation.points.length);
        fixation.y = (y / fixation.points.length);
        fixation.time = fixation.points[0].time;
        fixation.duration = this.calculateTime(fixation.points);
        if (fixation.points[0].curLine != -41) {
            let occurrences = fixation.points.reduce(function (acc, curr) {
                return acc[curr.curLine] ? ++acc[curr.curLine] : acc[curr.curLine] = 1, acc
            }, {});
            fixation.lineAns = Object.keys(occurrences).reduce((a, b) => occurrences[a] > occurrences[b] ? a : b);
        }
        return fixation;
    }

    getSetting() {
        return this.duration + "," + this.dispersion;
    }
}

/**
 * 
 * @param {Point[]} data 
 * @param {Number} dur 
 * @param {Number} dis 
 * @returns {Fixation[]}
 */
export function generateIDT(data, dur, dis) {
    let fix = new IDT(dur, dis);
    return fix.generateFixation(data);// fixation in type Fixation
}

Vertical drift

K-cluster.js

This is a script to handle vertical drift by categorize all points to K-clusters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

import { Point } from './IDT.js'


// Should be used to guess lines instead of generating fixation pattern

const kMeans = (data, k = 1) => {
    const centroids = data.slice(0, k);
    const distances = Array.from({ length: data.length }, () =>
        Array.from({ length: k }, () => 0)
    );
    const classes = Array.from({ length: data.length }, () => -1);
    let itr = true;

    while (itr) {
        itr = false;

        for (let d in data) {
            for (let c = 0; c < k; c++) {
                distances[d][c] = Math.hypot(
                    ...Object.keys(data[0]).map(key => data[d][key] - centroids[c][key])
                );
            }
            const m = distances[d].indexOf(Math.min(...distances[d]));
            if (classes[d] !== m) itr = true;
            classes[d] = m;
        }

        for (let c = 0; c < k; c++) {
            centroids[c] = Array.from({ length: data[0].length }, () => 0);
            const size = data.reduce((acc, _, d) => {
                if (classes[d] === c) {
                    acc++;
                    for (let i in data[0]) centroids[c][i] += data[d][i];
                }
                return acc;
            }, 0);
            for (let i in data[0]) {
                centroids[c][i] = parseFloat(Number(centroids[c][i] / size).toFixed(2));
            }
        }
    }
    console.log(data)

    return classes;
};

console.log(kMeans([[0, 0], [0, 1], [1, 3], [2, 0]], 4))

export function KCluster(points, screenSizeVec, autocompleteLoc) {
    // TODO: not necessary now
}

attach.js

This is the baseline algorithm for fixing the vertical drift for eye-tracker. Previous research use this as a baseline while also have a good performance.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import { Point, Fixation } from './IDT.js'

/**
 * 
 * @param {Fixation[]} fixations 
 * @returns 
 */
export function attachFixation(fixations) {
    // set lines, actually, only fix negative offset and outofrange offset 
    // 303px with 30px per line, so 10 lines in total
    // 0-1 stands for the first line.
    for (let i of fixations) {
        if (i.y < 1) {
            i.line = 1;
        } else if (i.y >= 9) {
            i.line = 10;
        } else {
            i.line = (i.y | 0) + 1;
        }
    }
    return fixations;
}

Internal test script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289

import * as IDT from './IDT.js';
import * as IDTM from './IDT-proposed.js';
import { attachFixation } from './attach.js';
import { generatePattern, fixit } from './autoFix.js';
import { convertFile } from './readData.js';

let screenY = 864;
let dataFile = "./testData.txt"
async function simpleAttach() {
    let sourceSession = await convertFile(dataFile);
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            for (let k of Object.keys(sourceSession[j])) {
                let tmpPoints = [];
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; // line number start from 1
                    tmpPoints.push(newP);
                }
                source = source.concat(tmpPoints);
            }
        }
    }
    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;// divide by 30 to return to previous format
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;
    source = attachFixation(source);

    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        if (i.line == i.curLine) {
            tempResult["correct"]++;
        } else {
            tempResult["incorrect"]++;
        }
    }

    console.log("result:" + tempResult["correct"] + " " + tempResult["incorrect"]);
    console.log(tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]))
    return tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"])
}

async function IDT_perline_Attach(dur, dis) {
    // Need to get the best fixation working pair
    // fixation should be done on each session of data, categorize by autocompletion box position
    let sourceSession = await convertFile(dataFile);
    // in format {"boxY":{"line":[data]}, "screenY": 864}
    screenY = parseInt(sourceSession["screenY"])
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            for (let k of Object.keys(sourceSession[j])) {
                let tmpPoints = [];
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; 
                    tmpPoints.push(newP);
                }
                source = source.concat(IDT.generateIDT(tmpPoints, dur, dis));
            }
        }
    }
    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;// divide by 30 to return to previous format
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;
    // let pattern = generatePattern(source, screenY);
    // source = fixit(source, pattern, screenY);
    source = attachFixation(source);
    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        for (let j of i.points) {
            if (i.line == j.curLine) {
                tempResult["correct"]++;
            } else {
                tempResult["incorrect"]++;
            }
        }
    }
    console.log("dur:" + dur + "dis:" + dis + "result:" + tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]));
    console.log("Points: " + source.reduce((a, b) => a += b.points.length, 0))
    // return source.reduce((a, b) => a += b.points.length, 0);
    return [tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]), source.reduce((a, b) => a += b.points.length, 0)];
}


async function IDT_persession_Attach(dur, dis) {
    let sourceSession = await convertFile(dataFile);
    // in format {"boxY":{"line":[data]}, "screenY": 864}
    screenY = parseInt(sourceSession["screenY"])
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            let tmpPoints = [];
            for (let k of Object.keys(sourceSession[j])) {
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; // line number start from 1
                    tmpPoints.push(newP);
                }
            }
            source = source.concat(IDT.generateIDT(tmpPoints, dur, dis));
        }
    }
    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;// divide by 30 to return to previous format
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;
    // let pattern = generatePattern(source, screenY);
    // source = fixit(source, pattern, screenY);
    source = attachFixation(source);
    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        for (let j of i.points) {
            if (i.line == j.curLine) {
                tempResult["correct"]++;
            } else {
                tempResult["incorrect"]++;
            }
        }
    }
    console.log("dur:" + dur + "dis:" + dis + "result:" + tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]));
    console.log("Points: " + source.reduce((a, b) => a += b.points.length, 0))
    // return source.reduce((a, b) => a += b.points.length, 0);
    return [tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]), source.reduce((a, b) => a += b.points.length, 0)];
}


async function IDTM_perline_Attach(dur, dis) {
    let sourceSession = await convertFile(dataFile);
    // in format {"boxY":{"line":[data]}, "screenY": 864}
    screenY = parseInt(sourceSession["screenY"])
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            for (let k of Object.keys(sourceSession[j])) {
                let tmpPoints = [];
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; // line number start from 1
                    tmpPoints.push(newP);
                }
                source = source.concat(IDTM.generateIDTM(tmpPoints, dur, dis));
            }
        }
    }
    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;// divide by 30 to return to previous format
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;
    // let pattern = generatePattern(source, screenY);
    // source = fixit(source, pattern, screenY);
    source = attachFixation(source);
    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        for (let j of i.points) {
            if (i.line == j.curLine) {
                tempResult["correct"]++;
            } else {
                tempResult["incorrect"]++;
            }
        }
    }
    console.log("dur:" + dur + "dis:" + dis + "result:" + tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]));
    console.log("Points: " + source.reduce((a, b) => a += b.points.length, 0))
    return [tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]), source.reduce((a, b) => a += b.points.length, 0)];
}


async function IDTM_persession_Attach(dur, dis) {
    let sourceSession = await convertFile(dataFile);
    screenY = parseInt(sourceSession["screenY"])
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            let tmpPoints = [];
            for (let k of Object.keys(sourceSession[j])) {
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; // line number start from 1
                    tmpPoints.push(newP);
                }
            }
            source = source.concat(IDTM.generateIDTM(tmpPoints, dur, dis));
        }
    }
    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;
    source = attachFixation(source);
    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        for (let j of i.points) {
            if (i.line == j.curLine) {
                tempResult["correct"]++;
            } else {
                tempResult["incorrect"]++;
            }
        }
    }
    console.log("dur:" + dur + "dis:" + dis + "result:" + tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]));
    console.log("Points: " + source.reduce((a, b) => a += b.points.length, 0))
    // return source.reduce((a, b) => a += b.points.length, 0);
    return [tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]), source.reduce((a, b) => a += b.points.length, 0)];
}

async function getBestFixation(dur, dis) {// do pattern
    let sourceSession = await convertFile(dataFile);
    // in format {"boxY":{"line":[data]}, "screenY": 864}
    screenY = parseInt(sourceSession["screenY"])
    let source = [];// array of result fixations 
    for (let j of Object.keys(sourceSession)) {
        if (j.localeCompare("screenY") != 0) {
            for (let k of Object.keys(sourceSession[j])) {
                let tmpPoints = [];
                for (let i of sourceSession[j][k]) { // in format: x,y,time,curBoxloc,curLine : For fixation, curline is determined by majority point
                    let newP = new IDT.Point(i[0], i[1] * 30, i[2], i[3]);
                    newP.curLine = i[4]; // line number start from 1
                    tmpPoints.push(newP);
                }
                source = source.concat(IDT.generateIDT(tmpPoints, dur, dis));
            }
        }
    }

    let newSource = [];
    for (let i of source) {
        i.y = i.y / 30;// divide by 30 to return to previous format
        if (i.y > -400) newSource.push(i);
    }
    source = newSource;

    source = attachFixation(source);

    let pattern = generatePattern(source, screenY);
    source = fixit(source, pattern, screenY);

    source = attachFixation(source);
    let tempResult = { "correct": 0, "incorrect": 0 }
    for (let i of source) {
        for (let j of i.points) {
            if (i.line == j.curLine) {
                tempResult["correct"]++;
            } else {
                tempResult["incorrect"]++;
            }
        }
    }
    console.log("dur:" + dur + "dis:" + dis + "result:" + tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]));
    // return source.reduce((a, b) => a += b.points.length, 0);
    return [tempResult["correct"] / (tempResult["correct"] + tempResult["incorrect"]), source.reduce((a, b) => a += b.points.length, 0)];
}

async function report() {
    // duration starts from 2500 with 500 increment
    let bestPossible = [];
    for (let dur = 500; dur < 2000; dur += 50) {
        for (let dis = 2; dis < 80; dis += 2) {
            let resultRatio = await IDT_perline_Attach(dur, dis);
            bestPossible.push({ "dur": dur, "dis": dis, "result": resultRatio[0] * 100, "points": resultRatio[1] })
        }
    }
    bestPossible.sort((a, b) => {
        return b["result"] - a["result"];
    })
    console.log(bestPossible);
}

IDTM_persession_Attach(550, 30)

function calculateCorrect(result) {
    for (let i of Object.keys(result)) {
        result[i]["ratio"] = parseFloat(((result[i]["correct"] / (result[i]["correct"] + result[i]["incorrect"])) * 100).toFixed(2));
    }
    return result;
}

IDT visualization script contains some sensitive information. I can provide if requested.

Reference

Andersson, Richard, et al. “One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms.” Behavior research methods 49 (2017): 616-637.

Kliegl, Reinhold, and Richard K. Olson. “Reduction and calibration of eye monitor data.” Behavior Research Methods & Instrumentation 13.2 (1981): 107-111.

Carr, Jon W., et al. “Algorithms for the automated correction of vertical drift in eye-tracking data.” Behavior Research Methods 54.1 (2022): 287-310.

The Servlet part of managing, rendering, and connecting applications and front-end:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

package website;

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

/**
 * Servlet implementation class hello
 */
public class hello extends HttpServlet {
private static final long serialVersionUID = 1L;

private static final HashMap<String, Process> runner = new HashMap<String, Process>();
private static final HashMap<String, Long> runtime = new HashMap<String, Long>();

/**
 * @see HttpServlet#HttpServlet()
 */
public hello() {
super();
// TODO Auto-generated constructor stub
}

/**
 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse
 *      response)
 */
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
// TODO Auto-generated method stub
HttpSession session = request.getSession();
try {
if (runner.containsKey(session.getId())) {
// continue
String code = request.getParameter("code");
if (code.isEmpty()) {
response.setContentType("application/json");
PrintWriter out = response.getWriter();
out.print("");
out.flush();
} else if (code.trim().compareTo("13") == 0) {
if (runner.containsKey(session.getId())) {
runner.get(session.getId()).destroy();
}
runner.remove(session.getId());
runtime.remove(session.getId());
response.setContentType("application/json");
PrintWriter out = response.getWriter();
out.print("Exit, please close browser.");
out.flush();
} else {
Process p = runner.get(session.getId());
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
os.write((code + System.lineSeparator()).getBytes());
os.flush();
BufferedReader dis = new BufferedReader(new InputStreamReader(in));
String disr = "";
while (true) {
String newLine = dis.readLine();
if (newLine.trim().equalsIgnoreCase("end")) {
break;
}
disr += newLine + "\n";
}

response.setContentType("application/json");
PrintWriter out = response.getWriter();
out.print(disr);
out.flush();

runtime.put(session.getId(), session.getLastAccessedTime());
}
} else {
String javaHome = System.getProperty("java.home");
String javaBin = javaHome + File.separator + "bin" + File.separator + "java";
File f = new File(MyPlayer.class.getProtectionDomain().getCodeSource().getLocation().getPath());
String className = MyPlayer.class.getName();
List<String> command = new LinkedList<String>();
command.add(javaBin);
command.add("-cp");
command.add(f.getAbsolutePath());
command.add(className);
ProcessBuilder builder = new ProcessBuilder(command);

Process p = builder.start();
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
BufferedReader dis = new BufferedReader(new InputStreamReader(in));
String disr = "";
while (true) {
String newLine = dis.readLine();
if (newLine.trim().equalsIgnoreCase("end")) {
break;
}
disr += newLine + "\n";
}

response.setContentType("application/json");
PrintWriter out = response.getWriter();
out.print(disr);
out.flush();

runner.put(session.getId(), p);
runtime.put(session.getId(), session.getLastAccessedTime());
}
} catch (Exception e) {
if (runner.containsKey(session.getId())) {
runner.get(session.getId()).destroy();
}
runner.remove(session.getId());
runtime.remove(session.getId());
response.setContentType("application/json");
PrintWriter out = response.getWriter();
e.printStackTrace(out);
out.flush();
}

// remove all long running job
for (String a : runtime.keySet()) {
if ((session.getLastAccessedTime() - runtime.get(a)) >= 180000) {// 3 minutes
if (runner.containsKey(a)) {
runner.get(a).destroy();
}
runner.remove(a);
runtime.remove(a);
}
}
}

public void destroy() {
for (String a : runner.keySet()) {
runner.get(a).destroy();
runner.remove(a);
runtime.remove(a);
}
}

}

Nowadays, new computer components are introduced into the retail market annually. Customers may find that the abundant component choices make it difficult to choose the optimum combination of hardware components while building a new computer. This project aims to take in the main components of a computer hardware build and provide several of the highest-performing options that can meet a consumer’s budget.

Original PDF:

Extracted text:

Team Assignments

Team members

Maxim Edelson Shaokang Jiang Junhua Yang Jinghong Luo

Data Curation

Maxim is scrapping data using python and do completeness, correctness
checking of all data. Shaokang was scrapping data using Node.js and
autoscrapping feature. Both of them are discussing on how to formulate a
better, more meaningful data.

Linear Programming Formulation and Analysis

Maxim, Junhua and Jinghong are using python to formulate and analysis
the problem. All of us are working to test and revise the python
solution code and make it more feasible in running. Shaokang is using
GAMS to formulate and analysis the problem.

Paper Writing

Maxim did majority of introduction, problem setup and primal
formulation. Shaokang did majority of intended approaches, result and
conclusion. Jinghong did intended contributions and contribution vs
previous part. All people are working on the primal, dual, KKT
formulation and paper reviewing, editing.

Introduction

Motivation

Nowadays, new computer components are introduced into the retail market
annually. Customers may find that the abundant component choices make it
difficult to choose the optimum combination of hardware components while
building a new computer. This project aims to take in the main
components of a computer hardware build and provide several of the
highest-performing options that can meet a consumer’s budget.

Previous Works

Linear programming is a modern, powerful mathematical technique that is
commonly used to solve optimization problems with constraints
[@bradley1971transformation],[@dantzig2002linear],[@sultan2014linear].
In particular, there are already many research publications that focus
on using linear programming to solve linearly constrained optimization
problems. By leveraging linear programming techniques, researchers and
practitioners can efficiently solve complex problems with multiple
constraints and objectives, enabling them to make more informed
decisions.

One example of using linear programming is the “diet problem”, which
seeks to find the optimal combination of foods to meet nutritional
requirements at the lowest cost. This problem can be formulated as a
linear programming problem, where the objective is to minimize the total
cost of the food, subject to constraints on the required nutrients.
Authors from paper [@paper1] aim to solve the diet problem of McDonald’s
sets menu in the hope of finding the optimal cost while satisfying the
daily calories and nutritional requirements for a person.

Intended Contributions

In this paper, we have explored state-of-art methods of applying linear
programming with constraints to help us formulate our problem and
analyze it using primal, dual, and KKT conditions. We came up with our
hardware component performance measurement formula in order to formulate
the optimization problem and set appropriate constraints based on
real-world scenarios. The novelty and main attraction of this paper is
applying integer programming techniques to solve computer building
problem, which is a relatively new and under-explored area in the
computer industry. We also compare the efficiency between commercial and
open source solvers.

Organization of the Paper

The rest of the paper is divided into different sections illustrating
our work. First, we go in-depth to discuss the problem statement in
section 3, demonstrating the reasoning behind our objective function and
constraints, as well as computing the dual formulation and KKT
conditions of the primal problem. In section 4, we talk about the method
we use to build the linear programming problem. Then we present the
results and conclude our work in section 5.

Contribution vs Previous Works

Even though the application of linear programming have been widely used
in various fields to solve complex optimization problems, its
application in the computer industry has yet received attentions on.
With this paper, we hope to make significant impact on the computer
community by offering this new way of identifying the computer build to
customers who are looking for cost-effective builds with optimal
performance. We think by exploring into the novel field of computer
building with linear programming makes us unique compared to the other
previous works.

Statement of the Problem

Problem Setup

Before formulating the primal problem, it is necessary to discuss how we
have arrived to the problem itself. In this paper, we utilized data from
the website UserBenchmark.com by web
scraping down component names, benchmarks, prices, samples per
component, and value for GPUs, CPUs, SSDs, HDDs, and RAM (automatically
updating). Using the datasets we collected, we crafted g(s) (equation
(1)), which is used as a part of our objective function (equation (3)).
This objective function allows us estimate the performance of a hardware
based on its price, benchmark, value of the component, and number of
benchmark samples.

$g(s) = \frac{\text{benchmark * value * samples}}{\text{price}}$

Formula (1) computes a ratio of performance to cost, adjusted by the
hardware’s overall value and the sample size used to obtain the
benchmark score. A higher score indicates better performance for the
price. It’s important to note that this formula is just one possible way
to measure hardware performance, and that there are many other factors
that may be relevant depending on the context.

Next, to define our constraints, we look back to our motivation of the
paper, which is to find highest-performing hardware build options that
can meet the budget. To do this, we need to 1) constrain the total
budget of all hardware combinations to be less than the fixed budget we
specified, and 2) limit the count of each hardware component in a
combination to 1 (i.e., one GPU, one CPU, etc…) because we are
looking for the optimum combination of hardware components in terms of
performance per cost.

Primal Formulation

$\text{From (1), let } g(s) = \frac{s_1*s_2*s_3}{s_4} \text{ where } s \in R^{4}$

In equation (1), $s_1$ is the component benchmark (with a range of
0-100), $s_2$ is the number of recorded benchmark samples min-max
normalized between 0-1, $s_3$ describes whether this component is
valuable from the user’s perspective (with a range of 0-100), and $s_4$
is the current component price with no upper bounds.\

Let $n \in \mathbb{N}$ be the number of components in each category
(CPU, GPU, SSD, RAM)

$\text{max } \sum_{i\in n}g(c_i)*x_{1i} + \sum_{i\in n}g(d_i)*x_{2i} + \sum_{i\in n}g(e_i)*x_{3i} + \sum_{i\in n}g(f_i)*x_{4i}$

$c, d, e, f \in \mathbb{S}^{n\text{x}4}$
$x_1, x_2, x_3, x_4 \in \{0,1\}^{n}$

subject to

$\sum_{i\in n}(c_{i,4}*x_{1i}) + \sum_{i\in n}(d_{i,4}*x_{2i}) + \sum_{i\in n}(e_{i,4}*x_{3i}) + \sum_{i\in n}(f_{i,4}*x_{4i}) \leq b \text{, where } b \in R_{++} \text{ is the budget}$

Dual Formulations

The Lagrangian function is $$\begin{split}
\mathcal{L}(x_1,x_2,x_3,x_4,\lambda,v_1,v_2,v_3,v_4)&=
\sum_{i\in n}g(c_i)*x_{1i} + \sum_{i\in n}g(d_i)*x_{2i} + \sum_{i\in n}g(e_i)*x_{3i} + \sum_{i\in n}g(f_i)*x_{4i} \
&+ \lambda (\sum_{i\in n}(c_{i,4}*x_{1i}) + \sum_{i\in n}(d_{i,4}*x_{2i}) + \sum_{i\in n}(e_{i,4}*x_{3i}) + \sum_{i\in n}(f_{i,4}*x_{4i}) - b) \
&+ v_1(\textbf{1}^T x_{1} -1) + v_2(\textbf{1}^T x_{2} -1) + v_3( \textbf{1}^T x_{3} -1) + v_4(\textbf{1}^T x_{4} -1), \text{ where } \lambda \geq 0
\end{split}$$

To compute
$\max_{x_1,x_2,x_3,x_4} \mathcal{L}(x_1,x_2,x_3,x_4,\lambda,v_1,v_2,v_3,v_4)$,
let us take $$\begin{pmatrix}
\frac{\partial\mathcal{L}}{\partial x_1} \\
\frac{\partial\mathcal{L}}{\partial x_2} \\
\frac{\partial\mathcal{L}}{\partial x_3} \\
\frac{\partial\mathcal{L}}{\partial x_4}
\end{pmatrix} =
\begin{pmatrix}
0 \\ 0 \\ 0 \\ 0
\end{pmatrix} \implies
\begin{cases}
\sum_{i\in n}g(c_i) + \lambda \sum_{i\in n}c_{i,4} + v_{1}\textbf{1}^T = 0\\
\sum_{i\in n}g(d_i) + \lambda \sum_{i\in n}d_{i,4} + v_{2}\textbf{1}^T = 0\\
\sum_{i\in n}g(e_i) + \lambda \sum_{i\in n}e_{i,4} + v_{3}\textbf{1}^T = 0\\
\sum_{i\in n}g(f_i) + \lambda \sum_{i\in n}f_{i,4} + v_{4}\textbf{1}^T = 0
\end{cases}$$
When $\lambda$ and $v_1,v_2,v_3,v_4$ satisfy these four equations, we
get\

The Dual function is $$\begin{aligned}
\text{min } -\lambda b - v_1 - v_2 - v_3 - v_4\
\text{subject to }
\begin{pmatrix}
\sum_{i\in n}g(c_i) + \lambda \sum_{i\in n}c_{i,4} + v_{1}\textbf{1}^T = 0\
\sum_{i\in n}g(d_i) + \lambda \sum_{i\in n}d_{i,4} + v_{2}\textbf{1}^T = 0\
\sum_{i\in n}g(e_i) + \lambda \sum_{i\in n}e_{i,4} + v_{3}\textbf{1}^T = 0\
\sum_{i\in n}g(f_i) + \lambda \sum_{i\in n}f_{i,4} + v_{4}\textbf{1}^T = 0\
\lambda \geq 0
\end{pmatrix}
\end{aligned}
\text{ where } c, d, e, f \in \mathbb{S}^{n\text{x}4}$$

KKT Conditions

Primal Constraints

\textbf{1}^Tx_1 - 1=0
\textbf{1}^Tx_2 - 1=0
\textbf{1}^Tx_3 - 1=0
\textbf{1}^Tx_4 - 1=0
$\sum_{i\in n}(c_{i,4}*x_{1i}) + \sum_{i\in n}(d_{i,4}*x_{2i}) + \sum_{i\in n}(e_{i,4}*x_{3i}) + \sum_{i\in n}(f_{i,4}*x_{4i}) - b\leq 0$

Dual Constraints

$\lambda \geq 0$

Complementary Slackness

$\lambda[\sum_{i\in n}(c_{i,4}*x_{1i}) + \sum_{i\in n}(d_{i,4}*x_{2i}) + \sum_{i\in n}(e_{i,4}*x_{3i}) + \sum_{i\in n}(f_{i,4}*x_{4i}) - b]=0$

Gradient of Lagrangian With Respect to x Variables

\nabla_{x_1} = \sum_{i\in n}g(c_i)+\lambda\sum_{i\in n}c_{i,4}+v_1\textbf{1}^T=0
\nabla_{x_2} = \sum_{i\in n}g(d_i)+\lambda\sum_{i\in n}d_{i,4}+v_2\textbf{1}^T=0
\nabla_{x_3} = \sum_{i\in n}g(e_i)+\lambda\sum_{i\in n}e_{i,4}+v_3\textbf{1}^T=0
\nabla_{x_4} = \sum_{i\in n}g(f_i)+\lambda\sum_{i\in n}f_{i,4}+v_4\textbf{1}^T=0

Intended Approaches

In the real world case, a user should be able to select the limitation B
value. In our run, we simply set the limitation of B as 2000. And
initially, we set all variables to be binary since one computer can only
have one components in each machine.

Quality of choice

Why you make choice

There are no previous academic works discussing how to optimally select
a computer component combination using linear programming algorithms. As
such, we selected to use the benchmark score, sample number, value
score, and price to craft what we felt was a reasonable objective
function. Our function could reflect the user’s perspective in taking
multiple pieces of data about a hardware component into account when
making a selection.

Why this is good

Under normal conditions, we ran the linear programming models under
strict constraints, equation (4), of equal to 1 and binary variables. To
prove the tightness of relaxation of our solution, we relaxed these
constraints to $\leq$ to 1 and nonnegative variables. After rerunning
our experiment with these relaxed constraints and receiving the same
solution distribution (all solution vectors contained only zeros and
ones, i.e., are binary vectors), we can confidently declare that our
model solution is tight enough in our context.

Since our problem was a linear programming problem, there would always
exist an optimal solution as long as the feasible region exists. Because
we are always trying to maximum the objective function, which guarantee
us to find a valid solution under the given limitation B. We also tested
the optimality based on the KKT conditions described in the previous
section. We also tried several different B setup, we are able to find a
proven optimal solution in all cases. Which proves the optimality of our
model.

Novelty

Difference from previous work

There is no work in the previous discussing how to select a computer
using linear programming.

Consumers in the market have to use some search engine and compare each
product with days of work previously[@james2006buying]. Right now, we
introduced a new way of thinking what to purchase and how to organize
the computer in a more efficient way. User could simply run our script
with the latest data to get the recommendation with seconds of work. In
addition to a traditional scrapping using static web fetching, we also
introduced a way to automatically scrapping data from website by using
github action. The script is set to run once per three days. And a
contemporary way to scrapping website by simulating user behavior to
prevent blocking of IP address by using puppeteer. In this way, user is
always able to use the latest data to get the optimization result. See
here for the auto
scrapping repository.

Our setup of combining the binary vector with the associate price also
improves computational time a lot comparing to some intuitive setup of
using each components price * each choice vector as the function when
calculating the limitation. This reduces the equations required to
handle from (Dimension of cpu * Dimension of gpu * Dimension of hdd *
Dimension of mem) to only one equation. In the actual experiment, this
could reduce computation time from many minutes to 0.016 seconds.

Computational complexity

Since our problem was a linear programming problem, previous
studies[@karmarkar1984new] have shown that any linear programming is
able to run in polynomial-time, so our model should be able to run under
polynomial-time. But, since our model only have one dimension in each
equation, the actual runtime should be close to O(n). Nevertheless, it
is hard to scale the overhead if we have to use mixed integer solver to
solve the problem. Fortunately, our model is always able to be converted
to a problem using linear programming solver[@raidl2008combining], which
guarantees our model to run under O(n). However, it is hard and not
necessary to measure a runtime like this. In the real world usage case,
running a model like this would spent far more time on context switch
and read data from memory instead of doing the actual computation. This
could also provide a possible reason that our real tests always show a
runtime of 0.016 seconds even with more data and more dimensions.

The memory space complexity is also O(n) where n is calculated by sum of
Dimension of cpu and Dimension of gpu and Dimension of hdd and Dimension
of memory.

Conjectured Results, Conclusion, and Future Works

Results

All models are running on an old laptop, which has 2.5 GHz CPU and 8 GB
memory. In our setup of limitation as 2000, with the dataset from
auto-scrapping result from
here. We could
reach the following result:

​ CPU GPU

          Intel Core i7-6700K                       Nvidia GTX 1060-3GB                                                            SSD                                       RAM

Corsair Vengeance LPX DDR4 3000 C15 2x8GB Samsung 970 Evo Plus NVMe PCIe M.2 1TB

The time complexity of using mip solver in GAMS is 0.016 seconds. Time
complexity of using lp solvers on GAMS could reach the same result with
time complexity of 0.01 seconds. For python setup, the average time
consumed for 10 runs is 0.065 seconds.
In a run with a more meaningful result with limitation as 400, we could
reach the following result:

​ CPU GPU

         Intel Core i5-10600K                       Nvidia GTX 1060-3GB                                                            SSD                                       RAM

Corsair Vengeance LPX DDR4 3000 C15 2x8GB Samsung 970 Evo Plus NVMe PCIe M.2 1TB

Time complexity stays the same as previous for GAMS. For python setup,
the average time consumed for 10 runs is 0.05 seconds.

By using our way, user is able to get the recommendation result in a
fraction of a second instead of manually comparing between various
components. We also observed the commercial solvers generally have a
better performance than open source solvers on python.

Conclusion

In this work, we developed a novel and efficient way in finding the best
possible computer components pairs for a user to select under a certain
budget. By introducing the benchmark, popularity, valuable and price, we
formulate the problem as a convex optimization problem which maximizes a
customized objective function with the consideration of all four
factors. Through experiments under different settings, we show that the
proposed convex optimization algorithm could run fast and efficiently
generate a good result. Comparing to the previous situation where user
have to spent tons of hours in selecting and comparing components when
composing a new computer, our way is much more efficient and our model
is suitable to run on any computer.

Future Works

Based on our current solution, possible future works may search for more
meaningful objective functions. This would require additional real world
analysis and people have to conduct some user studies to learn which
factors among price, benchmark, popularity, and value are most
important.

As part of improving this work, we would like to add more features for
improving user interaction experience. Considering the size of this
problem, another possible route is to build a static website by using
jsLPSolver and host it on GitHub. This would be a low cost way to make
general public get access to our study result and benefit a wider
audience.

As our current system automatically updates the data every three days,
we utilize the most up-to-date data when searching for the optimal
combination. That being said, using a more meaningful combination of
data from all up-to-date data may be in the user’s best interest as it
potentially allows for better and more meaningful combination results.

References

[1] Gordon H Bradley. “Transformation of integer programs to knapsack problems”. In: Discrete mathematics 1.1 (1971), pp. 29–45.
[2] George B Dantzig. “Linear programming”. In: Operations research 50.1 (2002), pp. 42–47.
[3] Richard James. “Buying a computer?” In: ITNOW 48.3 (2006), pp. 24–24.
[4] Narendra Karmarkar. “A new polynomial-time algorithm for linear programming”. In: Proceedings of the sixteenth annual ACM symposium on Theory of computing. 1984, pp. 302–311.
[5] Nurul Farihan Mohamed et al. “An Integer Linear Programming Model For A Diet Problem Of Medonald’s Sets Menu In Malaysia”. In: 2021.
[6] Günther R Raidl and Jakob Puchinger. “Combining (integer) linear programming techniques and metaheuristics for combinatorial optimization”. In: Hybrid metaheuristics: An emerging approach to optimization (2008), pp. 31–62.
[7] Alan Sultan. Linear programming: An introduction with applications. Elsevier, 2014.

Original GAMS code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

option limrow=0, limcol=0;

set cpu_model,cpu_para,gpu_model,gpu_para,ssd_model,ssd_para,ram_model,ram_para;

parameter
    cpu(cpu_model,cpu_para),
    gpu(gpu_model,gpu_para),
    ssd(ssd_model,ssd_para),
    ram(ram_model,ram_para);
    
    
$gdxin data\cpu.gdx
$load cpu_model = Dim1
$load cpu_para = Dim2
$load cpu = d
$gdxin

$gdxin data\gpu.gdx
$load gpu_model = Dim1
$load gpu_para = Dim2
$load gpu = d
$gdxin

$gdxin data\ssd.gdx
$load ssd_model = Dim1
$load ssd_para = Dim2
$load ssd = d
$gdxin

$gdxin data\ram.gdx
$load ram_model = Dim1
$load ram_para = Dim2
$load ram = d
$gdxin

scalar B /400/;
binary variables cpu_i(cpu_model),gpu_i(gpu_model),ssd_i(ssd_model),ram_i(ram_model);
variables obj;

equations max, cpu_to1, gpu_to1, ram_to1, ssd_to1, limit;

max..
obj =e= sum(cpu_model, (cpu_i(cpu_model)*cpu(cpu_model, "bench")*cpu(cpu_model, "sample")*cpu(cpu_model, "valuable"))/cpu(cpu_model, "price"))
        +sum(gpu_model, (gpu_i(gpu_model)*gpu(gpu_model, "bench")*gpu(gpu_model, "sample")*gpu(gpu_model, "valuable"))/gpu(gpu_model, "price"))
        +sum(ssd_model, (ssd_i(ssd_model)*ssd(ssd_model, "bench")*ssd(ssd_model, "sample")*ssd(ssd_model, "valuable"))/ssd(ssd_model, "price"))
        +sum(ram_model, (ram_i(ram_model)*ram(ram_model, "bench")*ram(ram_model, "sample")*ram(ram_model, "valuable"))/ram(ram_model, "price"));

limit..
sum(cpu_model,cpu_i(cpu_model)*cpu(cpu_model, "price")) + sum(gpu_model,gpu_i(gpu_model)*gpu(gpu_model, "price")) + sum(ssd_model,ssd_i(ssd_model)*ssd(ssd_model, "price")) + sum(ram_model,ram_i(ram_model)*ram(ram_model, "price")) =l= B;

cpu_to1..
sum(cpu_model,cpu_i(cpu_model)) =e= 1;

gpu_to1..
sum(gpu_model,gpu_i(gpu_model)) =e= 1;

ram_to1..
sum(ram_model,ram_i(ram_model)) =e= 1;

ssd_to1..
sum(ssd_model,ssd_i(ssd_model)) =e= 1;

model hw1_1 /all/;

solve hw1_1 using mip maximizing obj;
display cpu_i.l, gpu_i.l, ram_i.l, ssd_i.l, obj.l;

Browser scrapping code

A sample usage of puppeteer to scrap some simple information. It can avoid being blocked based on IP frequency.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

const puppeteer = require('puppeteer');
const urlencode = require('urlencode');
const HTMLParser = require('node-html-parser');
const core = require('@actions/core');
const UIDGenerator = require('uid-generator');
const uidgen = new UIDGenerator();
const fs = require('fs');

let browser;
var url = undefined;

function replaceAll(originalString, find, replace) {
    return originalString.replace(new RegExp(find, 'g'), replace);
}

async function mainFunction1(name) {
    // use try catch without timeout at here
    // const browser = await puppeteer.launch({ headless: true , args: [`--no-sandbox`, `--disable-setuid-sandbox`]});\
    let screenshot = false;
    let hrstart = process.hrtime();
    browser = await puppeteer.launch({ headless: true });
    const page = await browser.newPage();
    // await page.setDefaultNavigationTimeout(500000);
    await page.setViewport({ width: 1920, height: 1080 });
    await page.setRequestInterception(true);

    page.on('request', (req) => {
        if (req.resourceType() === 'image' || req.resourceType() === 'media') {
            req.abort();
        }
        else {
            req.continue();
        }
    });

    let hrend = process.hrtime(hrstart);

    core.info("Start the first page, browser initialized in " + hrend[0] + "s")

    hrstart = process.hrtime();
    try {// wait for 60 seconds
        await page.goto('https://' + name + '.userbenchmark.com/', { waitUntil: 'networkidle2', timeout: 60000 }); // wait until page load
    } catch (e) {
        core.error("Wait too long for login page, terminating the program")
        await browser.close();
        core.setFailed(`Action failed with error ${e}`);
        process.exit(1);
    }
    // await page.waitForTimeout(2000000);
    hrend = process.hrtime(hrstart);
    core.info("Finish loading the first page, first page loaded in " + hrend[0] + "s")
    core.info("Start to get " + name + " info");
    let message = await page.evaluate(async () => {
        function handleNumber(num) {
            if (num.indexOf("k") != -1) {
                return (parseFloat(num) * 1000).toFixed(0);
            } else if (num.indexOf("M") != -1) {
                return (parseFloat(num) * 1000 * 1000).toFixed(0);
            } else {
                return (parseFloat(num)).toFixed(0);
            }
        }
        let toRe = "";
        document.querySelector("[data-mhth='MC_PRICE']").click()
        let counter = 0;
        while (true) {
            counter++;
            if (counter >= 60) return "Error";
            for (let i = 0; i < 7; i++) {
                await new Promise(resolve => setTimeout(resolve, 1000 + Math.random() * 1500));
                window.scrollTo(window.scrollX, window.scrollY + 500 + Math.random() * 300);
            }
            for (let i of document.getElementsByClassName("hovertarget")) {
                let lists = i.getElementsByTagName("td");
                let tempStr = lists[1].innerText.split("\n");
                //name,price,sample,valuable,bench,bench_low,bench_high
                if (lists[lists.length - 1].innerText.trim().length == 0)
                    return toRe;

                if (tempStr[1].indexOf("$") == -1) {
                    toRe += tempStr[1].trim()
                } else {
                    toRe += tempStr[1].substring(0, tempStr[1].indexOf("$")).trim()
                }
                toRe += "," + lists[lists.length - 1].innerText.split("\n")[0].replace("$", "").trim().replace(",", "") + "," + handleNumber(tempStr[2].replace("Samples", "").trim()) + "," + lists[3].innerText.split("\n")[0].trim();
                tempStr = lists[4].innerText.split("\n");
                toRe += "," + tempStr[0] + "," + tempStr[1].split("-")[0].trim() + "," + tempStr[1].split("-")[1].trim() + "\n";
            }
            document.getElementsByClassName("pagination pagination-lg")[0].lastElementChild.firstChild.click()
        }
    });
    if (message.localeCompare("Error") == 0) throw new Error("Error happened");

    await browser.close();
    return message;
}

async function main() {
    try {
        let hrstart = process.hrtime();
        let requests = ["cpu", "gpu", "ssd", "hdd", "ram"];
        for (let i of requests) {
            let mainMessage;
            let count = 1;
            try {
                mainMessage = await mainFunction1(i);
            } catch (e) {
                core.error("Error happened: " + e + ", try again");
                count = count + 1;
                await browser.close();
                try {
                    mainMessage = await mainFunction1(i);
                } catch (e) {
                    core.error("Error happened: " + e + ", try again");
                    count = count + 1;
                    await browser.close();
                    mainMessage = await mainFunction1(i);
                }
            }
            let hrend = process.hrtime(hrstart);
            core.info("Done " + i + " in " + hrend)
            // console.log(mainMessage);
            if (!fs.existsSync('./data')) {
                await fs.mkdirSync('./data');
            }
            await fs.writeFile("./data/" + i + ".csv", "name,price,sample,valuable,bench,bench_low,bench_high\n" + mainMessage, function (err) {
                if (err) {
                    return core.info(err);
                }
                core.info(i + ".csv was saved!");
            });
        }
    } catch (e) {
        await browser.close();
        core.setFailed(`Action failed with error ${e}`);
        process.exit(1);
    }
}

main();