MySQL Query Analyzer - first impressions

I've now been playing with MySQL Enterprise Monitor in its latest guise (which includes the Query Analyzer) for a number of days. To be honest, this is pretty amazing stuff.

Once installed, all you have to do is point your applications to port 4040 on the database server instead of the default of port 3306 and in a few minutes you'll start to see the queries being logged in the Query Analyzer tab. You can sort them by database, by execution time, by execution count, or any of the other table headings in the output. You can also filter by partial query as well as a variety of other filters.

"Ho hum" I hear you say? Yes you could probably just take a look at the processlist on your db server to see what it is doing now, but the statistics Query Analyzer provides can be extremely powerful. Two examples:

#1 - Badly performing query

I installed Quan (Query Analyzer) and within about 20 minutes it became clear I had a query that was averaging 14 seconds to run. This query doesn't get run all that often, every 5 minutes or so, but it uses a table that is the most active table on our system.

OK, click on the query and it pops up a full view, showing the query with literals replaced by place holders. If you have Example Queries configured, you can check to see if there is an example showing all of the data, and if you have Example Explain configured, you will also be able to get a full breakdown of what indexes will be used in resolving the query.

With a few minutes of checking the query and fine tuning it I was able to get it down to 0.15 seconds on average. That is almost a 100 fold improvement. Yes, I could probably have done something about it earlier, as I knew it was sub-optimal, but seeing it showing at the top of the list of queries hogging resources lifts the sense of urgency.

#2 - Excessive executions of query

While looking at the first one I noticed another query starting to work its way up the list. This one was a relatively simple query, but was clocking up more than 2,000 executions a minute. Now unlike the first I wasn't all that sure where this query was being run, but using the 'full' display format I could look for that exact string in the codebase, and was quickly able to identify where it was being called.

A little more searching and I found that the query was within a loop where it would get exactly the same information each time, because the variable data was set outside of the loop! Fix that and it now runs around 10-15 times a minute.

I could have discovered this one without Quan. Probably. If I'd done a code review that could have taken days if not weeks. If I wasn't so bored with the code review that I managed to understand the implications of the query within the loop. Instead I was able to find and fix it within minutes, and then be able to verify that the problem was resolved within a few more minutes.

Think on this one for a minute. To display the particular page in question there were up to 200 unnecessary queries. That is a lot of running around in circles. I'm now going hunting for the author of that gem with a big stick - and hoping that it wasn't me :)

Quan should be in every DBA's toolkit. It is an absolute must.

If I get my act together I'll put up some before and after screen shots.

*DISCLAIMER: I am an employee of Sun Microsystems, part of the team responsible for the MySQL.com websites, so I am almost certainly biased. MySQL.com is a pretty busy website though and we like to think that it hits our software pretty hard.

4 comments

Comment from: Arjen Lentz [Visitor]  
Arjen Lentz

I agree that it’s nifty, and well done MarkM & team.
However, the proxy-based approach is problematic. Such instrumentation should be build-in or at least facilitated by the server.

Any extra tool, such as proxy, is an additional possible point of failure. And, in any case, overhead. In this case, I believe such overhead is unnecessary.

19/11/08 @ 15:04
Comment from: [Member]
aj

I’m not sure I agree Arjen. The point is that with proxy I can switch stuff into and out of quan very quickly without affecting the setup or adding any code into the server itself. I’d rather have the overhead outside of the server than in it, and optionally available. That way I can look at a site and think - hmm, that should be running faster, swap it to quan for an hour or so, look at the results, fix any problems then swap it back to the standard server, all without having to start or stop the server, only having to modify the port that the app talks to.

19/11/08 @ 15:44
Comment from: [Member]
aj

Just on getting data from Proxy, have you seen the interview with Mark on the DevZone?

Take a look down towards the bottom at the answer to the question “Are you exploring ways to use MySQL Query Analyzer without needing Proxy middleware?” where Mark states:

“We don’t want MySQL Proxy to be the only source of the metrics we are collecting. How you get this data into the Service Manager is pretty flexible and is based on REST. There are some other sources we want to explore. We would like to consume log data, we would like to consume the slow query log, or the general query log, even if it is sampled from time to time, so that you don’t have to use MySQL Proxy to get this information.

Also, there are ways to instrument the MySQL Connectors to get this information, or to instrument your application. This would add extra information to send along that would be useful for the MySQL Query Analyzer.

That said, from the feedback we are getting, MySQL Proxy works for most people”

19/11/08 @ 16:18
Comment from: Mark Callaghan [Visitor]  
Mark Callaghan

It it great that this won’t require the proxy because that would limit its use.

I can’t restart apps to use a different port to talk to the server. I expect many of us have that constraint.

I wouldn’t put the proxy in front of a critical server. That is just one more point of failure. And I have yet to see dbt2 or sysbench results that include the use of deployment. Is the proxy still single-threaded? Does it run Lua on every query and result?

20/11/08 @ 00:04


Form is loading...