subject: Why Is Deduping An Essential Backup Tool For Data Storage Centers [print this page] In computing, data deduplication or deduping refers to the act of removing redundant data from the database. In other words, it indicates removal of duplicate data from a database which otherwise would take up extra space of data storage device.
Deduplication, according to observers, is an emerging technique in data storage management. It helps in reducing data center maintenance costs as well as saves valuable time, which is crucial for data storage center management. Besides, deduplication helps companies engaged in direct marketing in eliminating duplicate entries from their master database. This helps in reducing the chance of unintentionally spamming the senders with the same promotional mails, which in turn could hamper their reputation.
Importance and prime benefits
* Before going any further, let's find out what data deduplication is all about. The concept behind deduping is simple. When writing data that have already been written before, instead of doing the writing task again, a shortcut is left to the original data. For example, if we assume that 150 people in a business organization are to be sent an email with the same mail attachment, occupying 2MB space each, then instead of sending the same attachment, a pointer to the original file (attachment) is left. Thus, instead of utilizing 300MBs of server space, only 2MB of space is utilized while the message is conveyed to all the people in the list.
* Deduplication can take place in 2 levels namely, file-level deduplication and block-level deduplication. File-level deduplication in essence involves the removal of identical files from within a database or across multiple systems. Block-level deduping, on the other hand, essentially involves identification of "repetitive patterns of zeros and ones" and uses "logical and variable-sized blocks" for identifying repetitions and then eliminates the redundant data. Block-level deduplication helps in eliminating extra space from the storage device and keeps databases free of junk data.
* Deduping helps in data compression, as well. This results in creating space in data storages, which can be used for storing data as well as for other purpose. Analysts opine that deduping helps in minimising wastage of data storage space. If used properly, deduplication can save up to 70 percent disc space.
In spite of its several plus points, there are people who fear that deduping might cause data loss and corruption of storage. The fear can be traced to the use of cryptographic algorithms, employed particularly in block-level deduplication, which if goes wrong may corrupt entire files and storage data. Although possibility looms, there's no reported instance of data corruption due to deduplication. On the other hand, big enterprises have benefitted from data deduplication, especially when there are requirements for processing bulk data. For example, in postal processing, eliminating duplicate addresses, analysing and evaluating customer potential through a unified approach and avoiding errors in data entry, comes across as the foremost requirement, deduping can be extremely helpful in this regard.
