Tech Support Satisfaction Testing
The objective of the given research was to compare the performance of technical support services of the 10 companies from the TOP-10 at webhostinggeeks.com.
Subject of Research
The companies that served as subjects for this investigation are:
In total, there were three stages of support evaluation:
January 23-30, 2012
February 13-19, 2012
February 27 - March 4, 2012
Each evaluation was conducted for 2 weeks and covered all week days (including weekends) and all daytime spans (first shift - 3-11 pm, 2nd shift - 11 pm - 7 am, 3rd shift - 7 am - 3 pm, GMT+2). Time spans were chosen on the basis of personal experience of a former tech support operator.
During the first two evaluation rounds, the schedule was distributed as follows.
- First week: contacting the tech support via voice calls, e-mails and livechat.
- Second week: evaluation and reporting the results.
During the third stage of evaluation, it was decided that the first day should be for tech support testing, and the second for writing reports and observations. This scheme proved to be more convenient and effective.
Calls were made using numbers stated on hosting companies sites:
Livechat operators were reached using the following links:
Livechat Example: HostGator
E-Mail messages were sent to addresses found in the contact info at each hosting company. If the address wasn’t stated, we used the contact form on the website.
Example of question and answer via e-mail - GreenGeeks (1/12/2012):
Could you please advise what plan is better to choose to settle a simple web site with hosting and what VPS plan is better to settle 100Gb of content?
Rules for Interviewers
During the evaluation the following rules were stated for our testing team:
- Strict adherence to schedule. This is needed for qualitative data processing and clarity of the experiment.
- All interviewers should clearly state the days and hours for tech support communication.
- Date and time format in the statistics tables should be standardized. It is also important to specify the day of the week.
- Chat logs should be saved with the the time and date of conversation.
- Specify the date in the table.
- If the date falls on a Saturday or Sunday, mark the cell red.
- The quality rating should be presented in numbers accordin to the "Evaluation Criteria" table.
- Additional comments should be made in the evaluation cell or in the 'Additional comments' column.
- Unusual cases should be submitted to a separate document: http://goo.gl/zB2fC
- The interviewer must provide links to all records of calls, even if no response was received.
- All e-mail messages and chat logs should be included in the table by interviewer, in separate tabs for each host.
- Do not ask several questions at a time, if they were scheduled separately.
A set of 15 web hosting related questions were addressed to the support services of the given web-based companies via three communication channels: telephone, email, and live chat. All replies were recorded, saved and evaluated according to a set of specially elaborated criteria.
Due to the fact that the channels of communication varied considerably (the resulting answer texts being oral and written) there was a necessity to expand the number of evaluation criteria for voice calls, as compared to email and live chat messaging. Yet, each criterion for each communication channel was evaluated to a 0-to-5 grading scale, 5 being the topmost evaluation, and 0 - the lowest possible. 0 points was also attributed in cases of complete lack of answers from the support service, for instance, when the calls were not returned at all, the caller being left to listen to an audio message suggesting to try and ask their question via live chat.
A comparison of evaluation criteria for each channel of communication can be seen in the table below:
|Voice Calls||Live Chat||Emails|
Now we’ll study the meaning of each criteria in detail.
- speed of reply - how quickly the client is connected to a support service specialist for a live voice dialogue
- competence - the relevance and appropriateness of a support specialist's answer from the technical point of view
- clarity of instructions - how easy it is to understand the directions given by the support specialist
- politeness and patience - the degree of politeness and reserve shown by the support specialist
- salutations - greeting formulas used to open the conversation with the client and introduce the company
- active listening, additional qs - verbal signs of sustaining a comprehensive dialogue and additional questions required to understand the client's issue better
- summarizing and paraphrasing - use of summarizing and restating explanations or questions
- thanks and farewells - thanking and good-bye formulas used to close the conversation with the client
- fluency - seamlessly flowing speech, coherent ideas
- intonation - voice modulations required to keep a light and interested tone
- correctness - linguistic accuracy of grammar and vocabulary used by the support specialist
- pronunciation - natural and correct articulation of English sounds
- dynamics of conversation, pauses - the tempo at which the conversation develops, presence/absence of pauses
- coherence - logical connection of ideas expressed by the support operator
The main parameter that influences the quality of the tech support service of web hosting companies is the human factor. Depending on time of the day or question complexity, a support operator may respond with different speed, use template or personalized answers. For instance, when asked about the possibility to have unlimited hosting some operators answer “yes”, although the terms and conditions of the company indicate a limitation in this respect.
After having performed this three stages of researching into web hosting support quality, it could be concluded that the least preferred and “taken care of” communication channel is the email. Quite a number of emails in all companies were simply not answered, and the reason for this stays unknown. In other cases, parts of emails are ignored and just not answered. Apparently, the best and most efficient channel to address a web hosting company is the live chat.
The 10 web hosting companies were asked 15 questions via 3 channels of communication which should have produced 450 answers. Yet, in fact 127 answers were not received due to unknown reasons. 323 replies were received, as planned.