I started this investigation thanks to an article over at PPC Hero in which an AdWords representative stated that “The moment you load a keyword into AdWords, it receives a Quality Score. This initial Quality Score is determined by each keyword’s performance history for other advertisers.”
Later PPCPROZ also published a follow up email from the AdWords team stating that “both factors [the historic performance of the keyword for other advertisers and the specific quality of the account structure and landing page] are weighed initially at account creation, and that performance data will be taken into account as soon as the ad starts to serve.”

I believed the weighting that the AdWords reps seemed to be putting on this historic performance factor was too high. In my comment on the blog I made an assumption that relevancy, both between the keyword and the ad and the keyword and the landing page, would be the number one most important factor in determining a keywords initial quality score. I did this because the effect relevancy can have on a keyword’s CPC is quite dramatic. Competitor data would, I thought, be used be used as a metric to give the ads an “average” QS within the landscape. I decided to run an experiment to investigate initial Quality Score (IQS) and how it was affected by relevancy. It proved my assumptions to be very wrong indeed!

Methodology – the casual observer and the easily bored should feel free to read just the text in bold and my final observations.

Notes: Where I refer to Quality Score I am of course talking about the AdWords interface’s Visible Quality Score.
          Multiple match types were added of each keyword, but these made no difference – the visible QS was the same for a keyword no matter its match type. I have removed the multiple match type entries for ease of reading.
          All keywords were set up with a max CPC of £3.00, this factor should be irrelevant to the QS.

In order to test initial Quality Scores I set up a new campaign in a new account (so that there would be no issues with residual QS from other campaigns or adgroups). To begin with I set up 4 ad groups, each with one ad, the destination page of each ad being www.guava.co.uk.

The first ad was highly targeted to both the keywords I was using and the landing page. Almost every word in the ad appears on the landing page, and I even included a phrase from the ad directly in the keyword list.

Ad A:                                                      Keyword set 1:
Guava UK Services                                guava
We search online to find our expert            guava uk
clients a fresh Guava in the UK                ”fresh guava in theuk”
www.guava.co.uk/guava                            guava search
                                                              guava clients

All of the keywords came back with visible quality scores “OK”.

The second adgroup used exactly the same ad but this time had keywords focussed at the landing page content and deliberately not any of the copy in the ad. I actually took keywords suggested to me by the AdWords interface as being relevant to my landing page in the “sample keywords based on a scan of your site” section.

Keyword Set 2:
ppc
pay per click
pay per click advertising
seo
search engine optimisation
search engine optimization
web design

Whilst a majority of these keywords came back “ok”, the SEO keywords actually had a “poor” visible quality score. Landing page content and its relevance to your keywords obviously has very little if any affect on IQS.

The remaining two adgroups both had the same ad, which bore no relationship to the landing page content. One adgroup had keywords relevant to the ad, but obviously not the landing page, and one with keywords relevant to neither the ad nor the landing page (nor each other).

Ad B:
Adopt a Penguin
Will you care for a baby penguin?
Literally pick up a penguin today!
www.guava.co.uk/penguin

Keyword Set 3:                Keyword Set 4:
Penguin                            loft insulation
baby penguin                    tennis racket
penguins                           kite flying
adopt a penguin                trips to mars
                                        norwegian translation
                                        stop snoring

The keyword ‘adopt a penguin’ received a “great” quality score. Was this because of ad relevancy?
No – the keywords ‘loft insulation’, ‘trips to mars’ and ‘stop snoring’ also received a “great” quality score. This is interesting because according to Google a “great” QS means “The keyword is very relevant and may have a high click-through rate (CTR), relevant ad text and a unique, relevant landing page.” Well, they don’t have relevant ad text or landing page content, so they must have high “historic average” CTRs.
In fact these weren’t all random keywords, I know that ‘loft insulation’ and ‘stop snoring’ are both high competition terms, with good CTRs, so we must assume that it’s this that makes the difference.

It seems relevancy of the landing page and ad copy to the keyword has no real effect on initial Visible Quality Score.
Does this mean ALL the IQS is a result of historic keyword performance data? I couldn’t turn down the opportunity to see what happened with some more ‘historically’ top performing keywords. I selected half a dozen high volume, high demand (and so presumably high CTR) keywords and set them up with relevant and irrelevant ads as below.

Keyword Set 5:
cheap flights
spread betting
credit cards
mobile phone
car insurance
used car

Ad C:                                                Ad D:
Cutlery Warehouse                          Spread Betting Insurance
Save on your fork and spoon bills        Spread the cost of car insurance.
with a spork. Its scoopy & pointy!        Mobile phone insurance with credit
www.guava.co.uk/sporks                    www.guava.co.uk/cheap_credit

The keyword-rich nonsense in Ad D had no effect. In both ad groups the QS for almost every keyword was the same - “OK” for spread betting, “Great” for everything else.
The exception was that ‘car insurance’ had an “OK” QS in Ad D’s group, but a “Great” QS in Ad C’s group.  In the same campaign the keyword ‘car insurance’ has a worse QS when associated with an ad containing the term “Car Insurance” than it does when associated with one about cutlery!

Relevancy does still have one effect at this early stage of a campaign, that of CPC.
In our irrelevant ad campaign Spread Betting had a min first page CPC of £6.50 and Car Insurance a min first page bid of £5.75. In the ad group with the more relevant ad these were both £0.50 cheaper.
CPC should be tied up with QS. Back in the days of minimum bid we were told that “minimum bid is a direct reflection of Quality Score” to the extent that your visible quality score rating could be extrapolated by looking at your minimum bid. Obviously this is not the case anymore.

Final Observations

The difference in CPC demonstrates that AdWords does look at your ads and take the relevancy of keywords and ad copy into account. However it doesn’t seem to have any effect on the initial visible quality score attributed at keyword level. It seems this is almost entirely down to the historic CTR of that keyword. This is a big lesson for me. In a situation where I had uploaded a keyword and it was given a “poor” IQS I might well have looked at the ad and landing page to tried and improve this. This isn’t the right tactic at all – indeed it’s a waste of time – you might find you’re better off getting out there with an aggressive bid and generating a good CTR to wipe the historic QS and hopefully keep your bids lower in the long-term.

It is evident the affect of relevancy on Quality Score is not reflected at all through the visible Quality Score column at the point of keyword upload.