Numeric is expensive type - try to use float instead, maybe double.
If I am following the OP correctly the table itself has all the columns declared as varchar. The data in the CSV file is a mix of text, date and numeric, presumably cast to text on entry into the table.
But a CSV *is* purely text - no casting to text is needed. Conversion is only needed when the strings in the CSV are text representations of *non*-text data.
I'm guessing that the OP is using all text fields to deal with possibly flawed input data and then validating and migrating the data in subsequent steps. In that case, an ETL solution may be a better approach. Many options, both open- closed- and hybrid-source exist.