In a data transformation project, you identify a significant amount of duplicate data that needs to be resolved. Describe the steps you would take to identify and resolve these duplicates, including the tools and techniques you would use.