Comments on: The perfect compression algorithm http://diovo.com/2010/01/the-perfect-compression-algorithm/ Startups, Programming and stuff... Sun, 20 May 2012 07:14:44 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: clemjon gubaton http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-24611 clemjon gubaton Sat, 19 May 2012 03:33:19 +0000 http://www.diovo.com/?p=1442#comment-24611 i am very much interested about data compression... hope you'll share it sir.. i want to work on it and be my thesis sir... thank you!! i am very much interested about data compression… hope you’ll share it sir.. i want to work on it and be my thesis sir… thank you!!

]]> By: Niyaz http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-17233 Niyaz Fri, 06 Jan 2012 03:24:19 +0000 http://www.diovo.com/?p=1442#comment-17233 That is a nice idea. Be prepared to wait a few years to get your uncompressed data though. That is a nice idea. Be prepared to wait a few years to get your uncompressed data though.

]]> By: 5yrsTryingAndCounting http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-17219 5yrsTryingAndCounting Thu, 05 Jan 2012 22:12:50 +0000 http://www.diovo.com/?p=1442#comment-17219 In the scenario above of 10200: assume you have a file, when read as 1's and 0's it magically happens to correspond to the decimal number 100... 100*100 = 10000 the next possible number would be 101. 101*101 = 10201 10201 - 10000 = 201 possibilities... so starting w/ 10000, md5 it. no good iterate so you have 10001. md5 it no good ... .etc etc etc.... 10200, md5 it -> BINGO! as it would pertain to file compressions... start w/ an original file (the one to be compressed). MD5 it and store that key in a meta-data file. interpret it as a bit sequence (a really really really really big "number"). int_sqrt it. store that new (much smaller number) in a "compressed" file. when you need to de-compress, read "compressed" file and interpret as a really really really big number (1 less 'really' ;-) ) . iterate it to find the upper bound on what it can't be. then: square = bigNumber * bigNumber md5 square. no good iterate square md5 square. no good. etc.. etc.. etc... iterate square md5 square -> BINGO! YAY! In the scenario above of 10200:

assume you have a file, when read as 1′s and 0′s it magically happens to correspond to the decimal number 100…

100*100 = 10000

the next possible number would be 101.

101*101 = 10201

10201 – 10000 = 201 possibilities…

so starting w/ 10000, md5 it.
no good
iterate so you have 10001.
md5 it
no good
… .etc etc etc….

10200, md5 it -> BINGO!

as it would pertain to file compressions… start w/ an original file (the one to be compressed). MD5 it and store that key in a meta-data file. interpret it as a bit sequence (a really really really really big “number”). int_sqrt it. store that new (much smaller number) in a “compressed” file.

when you need to de-compress, read “compressed” file and interpret as a really really really big number (1 less ‘really’ ;-) ) . iterate it to find the upper bound on what it can’t be. then:

square = bigNumber * bigNumber
md5 square.
no good
iterate square
md5 square.
no good.

etc.. etc.. etc…
iterate square
md5 square -> BINGO! YAY!

]]> By: 5yrsTryingAndCounting http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-17217 5yrsTryingAndCounting Thu, 05 Jan 2012 22:01:21 +0000 http://www.diovo.com/?p=1442#comment-17217 Here's a fun thought for you all.... in the int_sqrt scenario... why store the error? so you end up w/ a set of "candidate" bit sequences... if you hash the original bit sequence beforehand (say MD5) and also record the original bit sequences' length then you'll confine the possible permutations reasonably enough to brute-force MD5 each of the "candidate" bit sequences. In the case of MD5, your limit case would be 16 bytes (the 'new' MD5) + int_sqrt( previous iteration and its MD5) w/o doing the math on that limit case, I'd guess it to be around 32bytes... not that big... Trade-off: since the "compression" is lossy, you trade off filesize for computation time to pick the correct "candidate" bit-sequence. (we have really fast computers now ;-) ) Here’s a fun thought for you all…. in the int_sqrt scenario… why store the error? so you end up w/ a set of “candidate” bit sequences… if you hash the original bit sequence beforehand (say MD5) and also record the original bit sequences’ length then you’ll confine the possible permutations reasonably enough to brute-force MD5 each of the “candidate” bit sequences.

In the case of MD5, your limit case would be 16 bytes (the ‘new’ MD5) + int_sqrt( previous iteration and its MD5)

w/o doing the math on that limit case, I’d guess it to be around 32bytes… not that big…

Trade-off: since the “compression” is lossy, you trade off filesize for computation time to pick the correct “candidate” bit-sequence.

(we have really fast computers now ;-) )

]]> By: 5yrsTryingAndCounting http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-12542 5yrsTryingAndCounting Thu, 10 Nov 2011 17:41:24 +0000 http://www.diovo.com/?p=1442#comment-12542 Although the original idea of (charting the "position" of a bit and finding a mathematical 2-D function to represent the data, such that each "whole number" in the formula corresponds to a point on the graph) might still work... but it bears a whole lot more thinking.... also, if that can be shown to have even marginal success, then how about trying to chart everything in 3-D or 4-D (or n-D) and creating a function that "hits" all those spots in 'sequence' ... Best of luck to anyone out there w/ enough time/experience/knowledge/interest to pursue this approach! Although the original idea of (charting the “position” of a bit and finding a mathematical 2-D function to represent the data, such that each “whole number” in the formula corresponds to a point on the graph) might still work… but it bears a whole lot more thinking…. also, if that can be shown to have even marginal success, then how about trying to chart everything in 3-D or 4-D (or n-D) and creating a function that “hits” all those spots in ‘sequence’ …

Best of luck to anyone out there w/ enough time/experience/knowledge/interest to pursue this approach!

]]> By: 5yrsTryingAndCounting http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-12541 5yrsTryingAndCounting Thu, 10 Nov 2011 17:34:41 +0000 http://www.diovo.com/?p=1442#comment-12541 Just thought I'd comment back here. After talking with someone who's familiar with Fourier Series, this won't work. Fourier Series require repeating patterns and even then, only on pre-determined harmonics... Still might be possible to represent a bit sequence as a harmonic frequency, but it's most likely to cause a data-explosion such that the outcome is a larger file than the file itself.. :-( Just thought I’d comment back here. After talking with someone who’s familiar with Fourier Series, this won’t work. Fourier Series require repeating patterns and even then, only on pre-determined harmonics… Still might be possible to represent a bit sequence as a harmonic frequency, but it’s most likely to cause a data-explosion such that the outcome is a larger file than the file itself.. :-(

]]> By: 5yrsTryingAndCounting http://diovo.com/2010/01/the-perfect-compression-algorithm/comment-page-1/#comment-10880 5yrsTryingAndCounting Fri, 21 Oct 2011 15:18:35 +0000 http://www.diovo.com/?p=1442#comment-10880 Well, I gotta admit, I'm still churning this one over... From what I can determine, the most likely way to 'solve' this would be to create some sort of algorithm that _describes_ the original data... don't try to think of it in any other way. That Magic Algorithm would probably do one thing on the compression end: create another algorithm that could be 'executed' and would spit out the original file... I don't know enough math to do this, so there's a large level of reasonable doubt, but has anyone ever thought of charting a binary sequence on a graph? 1 goes up, 0 goes down (y-axis) as position through the sequence goes from 0-n on the x-axis where n is the length of the sequence... once that is done... can an equation be made to describe those intersection points? if so... then possibly use some higher math functions (analog concepts, not digital) like a Fourier Analysis or something to break out its component parts..? This would sort of 'transition' this digital problem into the realm of analog signals processing... where a lot of 'tricks' can be played on the "data" ... just a thought for any of you geniuses out there who might have some tricks up your sleeves... if you do figure a way out... please keep it OpenSource under GNU Licensing. Thanks. Well, I gotta admit, I’m still churning this one over… From what I can determine, the most likely way to ‘solve’ this would be to create some sort of algorithm that _describes_ the original data… don’t try to think of it in any other way. That Magic Algorithm would probably do one thing on the compression end: create another algorithm that could be ‘executed’ and would spit out the original file… I don’t know enough math to do this, so there’s a large level of reasonable doubt, but has anyone ever thought of charting a binary sequence on a graph? 1 goes up, 0 goes down (y-axis) as position through the sequence goes from 0-n on the x-axis where n is the length of the sequence… once that is done… can an equation be made to describe those intersection points? if so… then possibly use some higher math functions (analog concepts, not digital) like a Fourier Analysis or something to break out its component parts..?

This would sort of ‘transition’ this digital problem into the realm of analog signals processing… where a lot of ‘tricks’ can be played on the “data” … just a thought for any of you geniuses out there who might have some tricks up your sleeves… if you do figure a way out… please keep it OpenSource under GNU Licensing.

Thanks.

]]>