BRZ MPG Data Analysis

[housecat]

For the past 4000 miles, I've been tracking my MPG and comparing it with the dash reading. Once I've collected enough data, I plan to do some sort of statistical analysis. I hope some of you might find this interesting.

Introduction

This experiment will measure the average amount of fuel consumed by the BRZ and compare it with the vehicles's measurement. The goal is to determine if the BRZ's mpg reading on the dash is within reasonable error. Later, the effects of different tires on fuel consumption may be tested.

Procedure for Measuring MPG

To measure the average mpg, first the trip odometer is reset to zero miles after the fuel tank is completely filled. During the next gas-station trip, the gas tank is refilled again completely. The amount of gallons displaced since the last fuel-up is measured by how many gallons it takes to fill the entire gas tank. The miles on the trip odometer keeps track of how many miles were driven with the consumed gasoline. By dividing the miles driven by the gallons of gasoline consumed, an average MPG measurement is obtained. The trip odometer is reset after refueling so another measurement can be made during the next gas station visit.

About the Vehicle Tested

The car driven is a 2014 Subaru BRZ Limited with a manual transmission. The vehicle is completely stock except for a K&N drop-in air filter. All MPG measurements made so far have been with the factory Michelin Primacy HP tires and 0W-20 synthetic motor oil. Only 91 octane fuel is to be used. Transmission and differential oils have been changed midway through the testing, and the date of the change is noted.

Sources of Error

Driving conditions such as wind, temperature, air oxygen content, incline, speed, traffic, and gasoline quality vary between measurements. However, these factors can be considered random. Thus, provided a sufficient amount of data (more than 20 measurements), the results are normalized and valid.

Furthermore, there may be error when refueling the vehicle. It is unknown if all gas station pumps fill the gasoline tank to the exact same amount every time. This error is small compared the the error from driving conditions.

While the vehicle is not being driven, it is possible that fuel in the gas tank evaporates away. This increases fuel consumption without gaining any miles on the odometer, meaning that an experimentally measured mpg would be lower than the vehicle's calculated mpg.

Preliminary Results

On average, the BRZ's dash reads a MPG measurement of 1.3 higher than the experimentally measured MPG. More data is required to verify this result. Later when I have time, I will determine the confidence of the result.


2-11-16 Update

I decided to start modding my car, so the experiment sorta went off on a tangent.

What I want to do here is provide the community with some data and give an idea of how certain mods might affect mpg. I'm doing this for fun and curiosity; I'm not trying to make statements like "this mod is bad because it decreases mpg". I couldn't care less if I had lower mpg, I want the power and handling from my mods! This isn't a published research paper or evidence for some court case. There are many factors that may affect mpg (temperature, altitude, humidity, road, incline, tire wear, oil wear, fuel quality, etc). I can't control them all or include every possible condition in my sample. Your results will vary.

Here's my updated data table:
https://docs.google.com/spreadsheets...P78/edit#gid=0


Please disregard the attached image, it's outdated but it seems I can't remove it.

[extrashaky]

Originally Posted by housecat

About the Vehicle Tested

The car driven is a 2014 Subaru BRZ Limited. The vehicle is completely stock except for a K&N drop-in air filter. All MPG measurements made so far have been with the factory Michelin Primacy HP tires and 0W-20 synthetic motor oil. Only 91 octane fuel is to be used. Transmission and differential oils have been changed midway through the testing, and the date of the change is noted.

All this tells you is how your car responds. Results from a sample size of one can't be extrapolated to a population, regardless of how many readings you take. I'm guessing your confidence interval is somewhere around 0%.

[Taimaishuuyo]

Pretty interesting. I wish I get this much. I usually average 19mpg

[BRZnut]

I do the same thing and have found the MPG reported by my BRZ runs ~1MPG over what I calculate. On average 28 around town and 40 on long trips.




I am surprise you are not higher with interstate driving. I usually get 39-42 MPG but that is when I drive 600 miles one on day.

[Stang70Fastback]

Originally Posted by extrashaky

All this tells you is how your car responds. Results from a sample size of one can't be extrapolated to a population, regardless of how many readings you take. I'm guessing your confidence interval is somewhere around 0%.

Well considering I've done the same thing with my car, and after each tank my car also had read exactly 1.5 MPG higher that what I had actually gotten, I think we can confidently say that the on-board computer is slightly on the optimistic side...

[gravitylover]

I've been tracking since I got the car (~15k) and it reads anywhere from 1 - 2.3 over calculated. I use the dash indicated avg just to get a feel for how well I'm doing but certainly don't/can't count on it.

[burdickjp]

I've been using fuelly since I bought my car. Every on board trip computer I've ever compared to has been more optimistic than what fully reports at each fill up.

[Quentin]

There are a ton of variables that impact this study. How worn are your tires? How accurate is the gas pump? How repeatable is the mechanism that causes the pump to stop from station to station? My read outs are all within 5% or so. That isn't big enough to get wrapped around the axle.



Sent from Tandy 400

[extrashaky]

Originally Posted by Stang70Fastback

Well considering I've done the same thing with my car, and after each tank my car also had read exactly 1.5 MPG higher that what I had actually gotten, I think we can confidently say that the on-board computer is slightly on the optimistic side...

Oh, I agree that the computer isn't accurate. I track my mileage using an app on my phone, and what's on the dash is usually about 2.5 to 3 mpg better than what the app calculates. It's very easy to prove the computer's calculation is unreliable.

What I'm saying is that he can't conclude that the computers in BRZs generally record 1.5 mpg higher than reality based on one car and one driver. He can only conclude that that outcome holds for him. If he wants to extrapolate to the entire population, he needs to select a desired confidence interval, calculate an appropriate sample size and test a sample large enough to generalize to the entire population.

How do we know from his experience that other people are not getting a 5 mpg difference? How do we know that some people aren't getting better mileage than the computer suggests?

The biggest concern I would have about relying on his numbers is that some people here have reported some rather low mpg readings from the dash without actually doing the math. I suspect that in at least some of those cases, the error is going the other direction, and they're actually getting better gas mileage than the car is telling them. But if they rely on his analysis, they'll think they're even worse off than they really are, because his analysis is only for his car and isn't applicable to their cars or driving habits.

Despite the quasi-scientific approach, this is still just anecdotal evidence at this point.

[travisb]

I have 14,617.1 miles on my 14 BRZ at the last fill up and 62 tanks of gas. 58 of those were at the same station and the vast majority at the same pump around 6:30am.

The dash indicates 1.17 higher on average than calculating how much it takes to "fill" the tank.

here's a quick summary
miles - 14617.1
gallons - 567.05
avg mpg - 25.78
$/g - $3.1805196
cost- $1,803.52

[housecat]

Thanks everyone for your input so far.

Originally Posted by extrashaky

All this tells you is how your car responds. Results from a sample size of one can't be extrapolated to a population, regardless of how many readings you take. I'm guessing your confidence interval is somewhere around 0%.

You're right, this is a problem. I was assuming that all 2014 model BRZs mpg calculating system are exactly the same, a reasonable assumption but not necessarily true. How about we just call this thread a case study? The results can be generalized until we find someone who does not share the same result.

Originally Posted by Quentin

There are a ton of variables that impact this study. How worn are your tires? How accurate is the gas pump? How repeatable is the mechanism that causes the pump to stop from station to station? My read outs are all within 5% or so. That isn't big enough to get wrapped around the axle.

Sent from Tandy 400

That's why I need a large sample size. Error in measurements and change in driving conditions is accounted for as stated in the post. The question I must answer is how much error is acceptable, 5%, 10%?

Originally Posted by BRZnut

I do the same thing and have found the MPG reported by my BRZ runs ~1MPG over what I calculate. On average 28 around town and 40 on long trips.

I am surprise you are not higher with interstate driving. I usually get 39-42 MPG but that is when I drive 600 miles one on day.

I forgot to include some information about this in the post. The speed on the interstate road I often drive on is 75mph and the road is mostly flat the entire way. The distance is about 175mi one direction. I also have a habit of going WOT on the entrance ramps The "mostly interstate driving" measurements may include 10 to 20 city miles.

[extrashaky]

Originally Posted by housecat

You're right, this is a problem. I was assuming that all 2014 model BRZs mpg calculating system are exactly the same, a reasonable assumption but not necessarily true.

That's not the only assumption you're making. You are also assuming that the error doesn't change with the driver or the type of driving. It could be that these things are exactly accurate for certain drivers and way off for others.

There is also a geographical element. People in different parts of the country are using different fuels. For example, it could be that the computer was calibrated using fuels available in California during the summer months, and during all other times of the year and in different locations the additive package throws off the calculation. You also have the problem that some people live in relatively flat places (like me), while others live in mountainous areas. Altitude will also make a difference, since 91 octane at 5000 feet behaves like 93 octane at sea level.

To do this right, you'd have to control for as many of these variables as possible. Or control for the big ones and then try to explain the outliers.

Originally Posted by housecat

How about we just call this thread a case study?

That's okay, but...

Originally Posted by housecat

The results can be generalized until we find someone who does not share the same result.

You already have people in this thread who haven't gotten the same result. Mine consistently says I'm getting 33-34 mpg on the highway, yet my phone app that's doing actual math and not estimating from fuel flow calculates it around 30.3 mpg.

So I still think the only generalization you can make at this point is that we can't rely on the mpg calculation from the car's computer.

[Stang70Fastback]

Originally Posted by extrashaky

Oh, I agree that the computer isn't accurate. I track my mileage using an app on my phone, and what's on the dash is usually about 2.5 to 3 mpg better than what the app calculates. It's very easy to prove the computer's calculation is unreliable.

What I'm saying is that he can't conclude that the computers in BRZs generally record 1.5 mpg higher than reality based on one car and one driver. He can only conclude that that outcome holds for him. If he wants to extrapolate to the entire population, he needs to select a desired confidence interval, calculate an appropriate sample size and test a sample large enough to generalize to the entire population.

How do we know from his experience that other people are not getting a 5 mpg difference? How do we know that some people aren't getting better mileage than the computer suggests?

The biggest concern I would have about relying on his numbers is that some people here have reported some rather low mpg readings from the dash without actually doing the math. I suspect that in at least some of those cases, the error is going the other direction, and they're actually getting better gas mileage than the car is telling them. But if they rely on his analysis, they'll think they're even worse off than they really are, because his analysis is only for his car and isn't applicable to their cars or driving habits.

Despite the quasi-scientific approach, this is still just anecdotal evidence at this point.

I think you're being unnecessarily anal about this... and this is coming from someone who is very anal about stuff. I'm an Aerospace Engineer. I know all about proper testing and data collection and assumptions and errors, etc... I don't think anyone here was suggesting that their number was a concrete indicator of how every single BRZ on the planet behaves. He was simply stating his numbers that he was collecting for himself as scientifically as possible. I added my numbers. Some people have added theirs. If enough people measure the difference and report back, then we will be able to get an overall idea. That's all we need.

We don't need everyone to go to the same stretch of the same road where we will install a dummy robot which will drive each car the exact same way, but only when the temperature is the exact same, and the wind is exactly the same, and the ground is the exact same temperature, and only after mounting the exact same tires, and taking out all of the extra custom modifications people have made, and re-installed OEM lighting so that the parasitic drag on the engine is the exact same...

...I could continue, but I think we all know that none of our cars is the same, and that none of these tests, no matter how detailed we try to be, are going to account for anywhere NEAR the proper amount of variables that might come into play. However, if enough people do the test over several tanks of gas, we will eventually get a pretty good idea of a general tendency of the on-board computer to over/underestimate fuel consumption.

[housecat]

Originally Posted by extrashaky

You already have people in this thread who haven't gotten the same result. Mine consistently says I'm getting 33-34 mpg on the highway, yet my phone app that's doing actual math and not estimating from fuel flow calculates it around 30.3 mpg.

So I still think the only generalization you can make at this point is that we can't rely on the mpg calculation from the car's computer.

That's the point. Maybe I wasn't speaking clear enough before and I apologize for that. I'm not testing the mpg and saying this is the mpg every BRZ should get (if that's what it sounded like I said). I'm finding the error between the dash's mpg and the experimentally measured mpg. I believe this error will be consistent regardless whether someone got 25mpg or 40 mpg. Obviously driving conditions will affect both the dash mpg and measured mpg, but I don't think driving conditions will affect the difference between the dash mpg and the measured mpg. The question I am answering is if all BRZ dash mpg readings are always overestimated (except for some outliers), and how much they are overestimated (seemingly about 4%). That answer seems to be obvious given the general consensus in this thread, but here's my experiment with data to prove it. Then my next question would be "is the BRZ wrong or are my measurements wrong?" Maybe the BRZ computer is right but some gasoline in the tank evaporates away while the car is parked.

To make this more interesting I will later use this average mpg data and compare it to my average mpg after changing tires, because I am interested in how tires can affect mpg. I will use a large sample data in attempt to normalize the data so that it can be compared without worrying about all of the variables. There may be issues such as "the new tires received more city driving than the Primacy tires did," or "the new tires were driven during a different season," and that's why I record dates and often take notes about my data. I could sift through the data and pick out only mpg measurements made from mostly interstate driving trips and compare those.

@Stang70Fastback extrashaky's criticism is constructive and good, it will help me perform a better experiment. However, I think there just may have been some miscommunication in what I'm trying to accomplish here.