Linux shell script or c programm that parses 1 file, finds duplicate lines and if there are duplicates takes some actions by fdagency

I need a shell script or C programm that will parse a file and will look for duplicate entries. What is that file : This file is a configuration file. Other scripts run and write this file, then these multiple configurations are concatenated in one single file… (Budget: $30-$250 USD, Jobs: C Programming, Linux, Software Architecture)