Tuesday, September 13, 2011

On Bicycle Reviews

Reunited with Patrizia
Once in a while I get an email from someone who bought a bicycle after reading positive things about it on this blog, only to discover that I also had other, more critical posts about the very same bike. When this happens it is only natural that the reader feels some degree of betrayal, and that I in turn feel guilty. I start to think that maybe I should wait until I've owned a bike for years before reviewing it, and that I should generally try to tone down the enthusiasm in my posts. But frankly, I don't think that would help matters. Having started this blog as a beginner, my preferences are in a constant state of evolution, making me an inherently untrustworthy reviewer. And I think the bigger issue is that all bicycle reviews are to some extent unreliable for these same reasons, and that reading them at face value is a mistake. Just consider the myriad of factors that can shape a bicycle review. Do you keep them in mind when interpreting the author's feedback?

Reviewer's physical characteristics
This one gets overlooked a great deal, but I think it's an important place to start. Consider, for example, that a cyclist's size and weight are going to affect their experience of any given bike. Unless you are similar in these characteristics to the reviewer, you may not experience the same bike in the same manner: a bicycle that feels perfectly comfortable to them may feel overly stiff (or overly flexible) to you; a bicycle that is perfectly proportioned for them may feel ill-proportioned to you.  Physical strength and degree of fitness plays a role as well. 

Reviewer's cycling background
How experienced is the reviewer at the time the review is written (that last bit is especially crucial to pay attention to when reading old reviews from bloggers who have since gained more experience)? And what type of cycling background are they coming from? An evaluation of a bicycle as fast/responsive by a seasoned racer is going to have very different implications than the same evaluation from someone whose experience has been limited to beach cruisers. Same with the notions of comfort, stability, and so on. 

Basis for comparison
What other bicycles has this person ridden and owned? If someone has never ridden a Dutch bike before and they review, say, a Batavus, their impressions are likely to be of Dutch bikes as a general concept rather of Batavus specifically, simply because the whole category is so new and striking to them. Same with racing bikes, mountain bikes, and so on. 

Duration of experience
Beware of statements such as "The bicycle felt great as soon as I started riding it," or "I knew right away that I loved it." And yes, I am probably guilty of making them myself - we all are. But the truth is that our impressions of bicycles change as we experience them across different contexts, and to experience them across different contexts we need time. How often and for how long has the reviewer been riding the bicycle? How long are their individual rides compared to yours? If a bicycle causes pain or fatigue after 50 miles, the reviewer whose rides are limited to 20 miles will be unaware of it. 

Applicability
Does the reviewer use the bicycle for the same purpose, or in the same way, as you would use it? The author's feedback is only truly applicable to the reader when that is the case. But if a bicycle is praised for loaded touring, whereas you are planning to commute on it or use it as a roadbike, chances are you will not experience its ride quality in quite the same way as the reviewer.

Value system
Every reviewer is different in terms of what it is they value about a bike. To some it is important that they feel no road vibration, whereas others could not care less. For some toe overlap is unacceptable, whereas others won't even notice it. Some care mostly about how a bicycle handles loaded, some have distinct seat tube angle preferences, some take note of how cleanly filed the joints of the frame are, some are concerned about the quality of the paintjob. There are loads of issues like this that a reviewer may omit simply because they don't care one way or the other, at least at the time of the review. 

Pattern of bias
Once you read a handful of reviews by the same author, a pattern of bias will usually emerge. Some reviewers criticise the heck out of all bicycles as a matter of course, so a "positive" review from them still looks pretty negative. Conversely, there are those who are enthusiastic about all the bikes they review, and then you have to carefully read between the lines and try to determine what they might be omitting. There are those reviewers who are prone to the "honeymoon effect" and those who keep a cool head. Reviewers' impressions can lean toward the emotional, the technical, the superficial, the overanalytical - you name it. Whatever their specific pattern may be, it holds clues for how to interpret that reviewer's feedback.

No doubt there are many more factors worth considering, and I invite you to share your own strategies for interpreting bicycle reviews. More than anything, I think it is crucial to read as many reviews of the same bike as you can find - and if a common thread emerges, that's when it becomes truly informative. Also, for those bicycles we don't get a chance to see in person, the images offered by reviewers can be more telling and detailed than those provided by the manufacturer. Reviews are usually useful, no matter how biased. But to "trust" a single reviewer because you like their blog or find their narrative style entertaining is, in my opinion, asking for trouble. There is no such thing as a reliable narrator, and bicycle reviews are no exception. 

35 comments:

  1. Fantastic post Veloria. I've noticed the same thing about bike parts. When finding reviews for new cassettes or deraileurs or chains, some of the things people say dont take into account their abuse.

    At the same time though, I want to read reviews by people who ride hard and are conscious of it. They seem more realistic.

    ReplyDelete
  2. Yes, I don't at all mean to suggest that reviewers ought to change their style, only that readers should be aware of the many layers of subjectivity they bring. But this same subjectivity is also what makes them alive and interesting.

    ReplyDelete
  3. I think it's very helpful to check out the opinion of others before purchasing pretty much any expensive item. However, even a well-written review is going to be a collection of value judgements. I think any reader should be aware of this and perhaps make sure they cross-reference reviews to get a broader spectrum of opinion.

    Even then, this should only be the first step taken in buying something as subjective as a bicycle. Which is why I still think there's something to be said for finding a LBS you can develop a comfortable relationship with and that will cater for your individual 'type' of cycling.

    Not wishing to sound harsh Velouria, anyone who purchases a bicycle on the strength of one review is probably asking for trouble and I certainly don't think that you should feel any guilt. Your reviews are always informative and interesting. However, I can't remember your having written in any of them, "Having read this, you should now leap out and spend your money based on my personal opinion!" Please keep reviewing . . . guilt-free :)

    ReplyDelete
  4. I think you do a fair job at reviewing bikes. You list the pros and the cons. I wouldn't worry about others' disappointmnnets. Before buying a bike the real test is to take it out for a reasonable length ride and to consult more than one review of the bike. Keep up the good job.

    ReplyDelete
  5. Love, or dislike, of a bicycle does evolve over time I have found and it is very difficult to do a review of a bike where that is not a huge influencing factor.

    This is a very thought provoking post and puts into words what I felt when writing a review of one of my bikes recently, I would hate for someone to read it and think that was the bike for them, then to buy it and find otherwise. Bike reviewing is a lot harder than it appears to be. It is all so subjective. Love of a bike can distort our perceptions of it as can a bad experience on a bike.

    From the reader's perspective, I also find it difficult to say whether I would buy a bike based on a review, yet it is naturally what we do when deciding which bike to buy. Reviews can sometimes leave more questions unanswered than they answer.

    ReplyDelete
  6. This is why I lament the ability to test drive bakfiets and trikes here in Boston. I would be extremely reluctant to buy an expensive bike without actually trying it, preferably for a couple of days.

    ReplyDelete
  7. Good points for all of us to consider. I guess it's why I think a reader has to take a blog post with a grain of salt {not at all saying your posts aren't valuable, but it's important to keep in mind that it is of course, your experience}. To me, it's a nice jumping off point to go and explore for myself, or it gives me a rough idea of what it might feel like... as you said though, it will be different for every person. I don't think that you {or any blogger} should feel guilty about sharing thoughts with others though. I know I wouldn't want to know that someone bought something solely based on my review, and I would likely have similar emotions to yours if that happened, but you honestly cannot be held responsible for another persons decision. Truly, blogs can often be one of the few ways someone is introduced to a particular style, brand, part, etc. I'm sure there are many who have been helped by your reviews (myself included), so I think it finds a way of balancing itself out.

    Thanks for an excellent reminder to us all. :O)

    ReplyDelete
  8. I seldom pay any attention to any bicycle reviews unless the reviewer notes a design or manufacturing concern that needs to be corrected to ensure the bicycles safety.

    I know none of us are joined at the hip so what we experience with any bicycle is unique to us alone. To think otherwise could get both disappointing and expensive.

    ReplyDelete
  9. Very accurate. I'd say the general level of experience and type of cycling are the major killers in people's preception/understanding of reviews.

    Yet, nothing prevents you from putting a little discalimer or "Please note" at the beginning of your review posts.
    It will not solve the problem, but at least people will know were you're (or were) at, at the time you wrote the review.

    It may not be obvious but the evolution you went through has been pretty major and unless someone is here on a daily basis...
    From beginning Pashleys to centuries and pacelines, I find there is ground to the comments you have received.

    ReplyDelete
  10. So what you are saying is that "your mileage may vary based on ...."

    Maybe you should include a link to this post to every review. That way, people who don't want to take ownership of their decisions can't pin it on you.

    ReplyDelete
  11. interesting! Yes I have read reviews of bikes that I have ridden and it's been interesting to see if I feel similarly or not to the reviewer.

    and also when reading your reviews I feel like we may percieve bikes somewhat similarly which is why I enjoy your take on things. Of course I still want your opinion on the bikes I have ridden most and I feel like that will tell me if we actually do perceive a bike the same....

    ReplyDelete
  12. There are many bicycles I could have ordered online based on positive internet reviews. However, all the reviews in the world can't substitute for a basic test-ride so if I couldn't find one in the flesh, I didn't consider it. This drastically cut down my bicycle choices but ultimately any review of any product online is always a guide rather than gospel and as consumers it's our responsibility to choose how informed we are prior to purchase. Doesn't matter if it's a toaster or a bicycle.

    ReplyDelete
  13. It was awfully wonderful of you to turn negative reader mail into an educational post, and not the ranting sort of "Take responsibility for your own damn decisions!" sort of thing. I would have been tempted to do the latter, at least inside my head. :)

    Thanks for taking the time to do such thorough reviews and writing a post like this.

    ReplyDelete
  14. Anon 12:21 - Good idea, I'll do that.

    ReplyDelete
  15. Cherilyn - I don't think it's negative per se, just human instinct. And I admit I've had similar feelings toward other blogs. I was disappointed when ecovelo sold their Pashleys for instance. I also think a lot of it has to do with people really wanting there to be the perfect solution, the perfect bike - whereas in reality such a thing does not exist.

    ReplyDelete
  16. Perfect bikes DO exist. I've owned 20 bikes, another 20 passed my hands briefly. Maybe another 20 ridden long enough for a review.
    The '73 Cinelli was perfect. The '63 Rickert is presently perfect. The Eastman DL-1 is near perfect. Some of the most imperfect - say the Moulton - evoke fond memories.
    Readers should learn to read. Writers need to write. This reader thinks V is a fine writer, and improving. If writers needed liability insurance in the event their readers did something foolish with their words, none would ever take up the craft.

    ReplyDelete
  17. I enjoy reading your reviews, V. and, as my mother would say, window shopping cost you nothing.

    Those who have owned one or several bikes have learned what does and doesn't work for them. For those who are in the market for a first bike, reading reviews and test riding each model can help them narrow down their choices, but it's in the owning, riding and caring for a bike that they will learn its strengths and shortcomings. I would advise anyone who is buying their first bike (or first in a long time) to start with a model that has been in production for a good number of years and has a track record of good reviews. I would also advise them not to break the bank on it, because once they learn what works or doesn't work for them, they will need to begin saving for a new bike.

    ReplyDelete
  18. "f writers needed liability insurance..."

    Oh god, what a terrible thought!

    ReplyDelete
  19. It's true enough that a cross-country biker won't even review a 3-speed city bike and that some blogs won't even consider reviewing less-than-5000$ bikes but there's also the problem of not finding any (serious) review anywhere on the web for some not particularly rare bikes ; I'm thinking of the "Marin Inverness" or the "Globe Roll", for instance.

    ReplyDelete
  20. I've noticed that as well. It's possible that people who ride "normal" bikes aren't sufficiently obsessive about bicycles to write reviews about them?

    ReplyDelete
  21. In fact I bought a Marin Inverness "fixed-gear" last week (my first bicycle since 30 years - I'm 65) and I was very satisfied with it for the first four days but then I started hearing strange metallic noises coming from the rear wheel. The mechanic re-aligned the wheel but the noise came back the next day ; I went back to the store and another mechanic re-aligned the wheel again. The same metallic noise came back the next day... After a polite discussion the owner of the store became impatient with me and in short he told me I was imagining things and that the bike was perfect. I phoned to Marin Bikes but no help from them since I live in MontrĂ©al. I searched the web but nobody seems to be experiencing this problem — or nobody writes about it.

    I sold the Inverness "as is" to a neighbor and now I was considering a Globe Roll 3 (I like pseudo-vintage bikes with white tires, etc.) but I didn't find any useful review on this bicycle. I'll take another chance with this one. Ainsi va la vie !

    (I hope you don't mind my somewhat being off-topic.)

    ReplyDelete
  22. Great topic! All of those considerations are really important for people to remember when researching bikes online. Personal reviews are so helpful during the initial research/obsession part of bike shopping, but can never replace one's own personal experience with a bike. The most helpful part of bike reviews for me is the opportunity to see more photos and get more info about a bike than is ever available from the manufacturer. Especially helpful for me when researching which city/Dutch bike to buy were photos of people actually riding the various bikes. Reviews provide great bonus material. :)

    ReplyDelete
  23. @ Giles

    Would you mind telling what store it was?

    ReplyDelete
  24. Dottie wrote "The most helpful part of bike reviews for me is the opportunity to see more photos (...)".

    Did you notice that the photos often are different from the actual bicycle, even on manufacturer's sites? For instance all colors are not available in all countries, the brakes or wheels may not be identical, the bicycle may come with a pannier or not, etc.

    ReplyDelete
  25. I think this is an outstanding post. In addition to pointing out important features of ANY bike review, you are also acknowledge that

    "Having started this blog as a beginner, my preferences are in a constant state of evolution, making me an inherently untrustworthy reviewer"

    I think that is a particularly important point for new readers. One of the paradoxes of Lovely Bike is that the site has all the visual hallmarks of having been written by an expert. It's very well written and illustrated. It is filled with links to "Bike Reviews" "Product Reviews" "Manufacturer's Profiles" etc. The site has attracted lots of commercial sponsors. New readers coming to the site see you designing bikes, doing test reports, collaborating with manufacturers, traveling to Interbike trade show. Because of all those trappings of expertise, many readers may not realize that you are actually a newish rider, and are still learning and experiencing many things for the first time yourself.

    Whether you want to be viewed as an expert or not, your highly successful web presence means you are SEEN as an authority. That increases the impact of what you write, but also means that people may put undue weight on your temporary opinions, which are themselves rapidly changing as you gain more experience. Reader (and writer) beware.

    ReplyDelete
  26. @ Montrealize

    I'll tell you the name by e-mail because I don't want to discredit this store for maybe my problem with them was an exception and not the rule.

    d r h g i l l e s @ m e . c o m

    ReplyDelete
  27. I agree with all the points you made about what needs to be taken into consideration when reading bike reviews. But, at the same time I can understand the readers sense of betrayal too. Often times, people like to tell others that so-and-so wrote a great review, which helped them with their decision to buy the bicycle. In the end, the way I look at it, if they still like the bike--despite your new opinion--then your opinion really shouldn't matter, as long as they are still happy with the bike. But, if they agree with your opinion, and realize the bike is not right for them... well, there's always Craigslist :).

    ReplyDelete
  28. Anon 3:08 - Very true, though to be fair I clearly state that I am a newish rider in the "About Me" page. If a reader thinks highly enough of this blog to value my opinions, surely they ought to at least glance at the About page - at which point they would notice this information.

    I also think that the whole beginner vs experienced rider dichotomy is somewhat misleading, in that *everyone's* preferences and opinions change over time - sometimes radically. When cycling experts give advice, it is based on their (or the industry's) understanding of a concept at the time. As new information becomes available, they will often alter their recommendation.

    Finally, just to point out that many people in the bicycle industry who do officially dispense advice have even less experience and knowledge than I do. I often overhear conversations in bike shops, where the salespeople or even mechanics feed customers misinformation. I also know framebuilders who got into cycling later than I did. They know how to build a particular kind of frame very well, but lack a wider base of knowledge.

    All in all I think it's important to keep in mind that blogs are neither manuals nor advice columns, but are usually documentations of one individual's experience/growth/journey/development/ whatever you want to call it. Any review posted on a blog should be interpreted in that context.

    ReplyDelete
  29. Big decisions, including purchases, are made irrationally. Buyers remorse is also emotional. How exhausting! No wonder they blame you. Even if you claim caveat emptor.

    ReplyDelete
  30. I don't want to make it seem like this is a frequent thing or like I get "blamed" per se. More like some people get nervous when I introduce ambiguity into ideas they were hoping would be more clear cut. Like, with my Rivendell - I thought it was fast a year and a half ago, and it was - compared to any other bike I'd ridden prior to it. Since then I've ridden racing bikes and it no longer seems like a fast bike. Some readers are unable to process this change in perspective, and they interpret it as my saying "never mind, the bike is not as good as I thought" even though that's not what I'm saying at all.

    Anyway, overthinking this stuff can make one paralised with doubt about expressing their impressions and opinions, which I think would be a shame. The way I see it, the reviewer's responsibility is to provide disclosure, while the reader's responsibility is exercise interpretation.

    ReplyDelete
  31. I learned from reading your blog that all types of bicycles and any given bicycle are a compromise and that people change over the years. That being said I still would value your opinion more than the opinion of many... because (but not only) you are never "pontificating" on any subject.

    ReplyDelete
  32. One key thing a review should provide is information. For example, toe overlap is a fact. The interpretation - whether it is a concern - is up to the reader. At Bicycle Quarterly, we try to provide information, and leave the evaluation to the reader. Everybody has different preferences, which is why we'd never select a winner in a "3-bike shootout" or crown a "Bike of the Year."

    Of course, every reviewer has preferences. There is no way around it. For example, we make no secret of the fact that we prefer bikes that respond well to spirited riding. If you share this preference, then Bicycle Quarterly's reviews will be most useful to you. However, even if you don't, you'll get useful information on whether the bike is well-made, or whether parts fall off over the course of our 200-mile test.

    ReplyDelete
  33. Your evolution is still too slow for me: I am waiting for you to tire of your Gazelle and put it up for sale....let me know if it is ever eclipsed in your affections by a younger, sexier bike.

    ReplyDelete
  34. : ) Nice try. The Gazelle is staying, particularly considering the things I've just seen at Interbike (the new Gazelles seem to be losing more and more lugs every season).

    ReplyDelete
  35. i think it's ok because your subjectivity is clearly part of it, it all reads like an experience, rather than some expert opinion. though, to be fair, i felt a bit queezy when i read a comparison of carradice saddlebags to some modern knock-off, and wondered where you were coming from. on the other hand, i just fitted some belleville bars and reverse levers to a bike after reading about it here, and it's just groovy..

    ReplyDelete