Question about the level of compression used in mysys/my_compress.c  
Index ‹ mysql
Author Message
mcallaghan
PostPosted: 2006-5-6 11:51:09 Top
mysql, Question about the level of compression used in mysys/my_compress.c Stewart Smith wrote:
> On Mon, 2006-05-01 at 17:59 -0700, Mark Callaghan wrote:
>> The function my_compress (in mysys/my_compress.c) uses the default level
>> of zlib compression when compressing network packets. For my
>> unscientific sample data 'gzip -1' is 2.5 to 3X faster than 'gzip' (the
>> default level of compression made the result file 25% smaller than the
>> fastest level). With a mysqld compiled to use the fastest level of zlib
>> compression, fetches of large amounts of data were faster when using the
>> fastest level of zlib compression rather than the default. These
>> results held even when the client and server were reasonably far apart
>> (ping time is 60ms).
>>
>> My belief is that compression of network packets is more likely to be
>> useful when the fastest level of zlib compression is used. Of course,
>> MySql clients with really high latency connections to servers or uses
>> where network traffic is significantly more expensive than server CPU
>> time may disagree.
>
> It sounds like it would be a good thing to make configurable. Possibly
> per connection as well :)
>
> I'm sure we'd be willing to help where needed if you wanted to write a
> patch.

OK. I will provide a patch soon. I will try to provide something that
has a global default and can be changed at the session level.

The current compression code allocates two buffers using free/malloc
that are each about as large as the uncompressed packet each time a
packet must be written to the network. I had another change that reduced
that to one call to malloc and free. That change did not have much of an
effect on performance, but I did not run any multi-threaded tests where
calls to malloc and free might be more expensive. I don't know how good
glibc malloc is for threaded apps. I do know that other mallocs are not
so good (Sun has published a paper on the benefits of mtmalloc for MySql).

From my limited understanding of MySql source, it tries to reduce the
number of calls to free and malloc, but the implementation of network
compression might be one case where that is not true.

Is it worth my time to try to prepare a patch that either reduces the
calls to malloc and free to one per flush of a packet to the network
rather than two calls as is currently done? The calls could be further
reduced by caching the allocated buffer as part of the thread context so
that it is only allocated once.

--
Mark Callaghan
email***@***.com

--
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe: http://lists.mysql.com/internals?unsub=email***@***.com

 
 
Index ‹ mysql