加入收藏 | 设为首页 | 会员中心 | 我要投稿 李大同 (https://www.lidatong.com.cn/)- 科技、建站、经验、云计算、5G、大数据,站长网!
当前位置: 首页 > 编程开发 > asp.Net > 正文

C#_.net core 3.0自定义读取.csv文件数据_解决首行不是标题的问

发布时间:2020-12-16 08:58:08 所属栏目:asp.Net 来源:网络整理
导读:linqtocsv文件有不太好的地方就是:无法设置标题的行数,默认首行就是标题,这不是很尴尬吗?? ?并不是所有的csv文件严格写的首行是标题,下面全是数据,我接受的任务就是读取很多.csv报表数据,里面就有很多前几行是说明性内容,下面才是标题和数据。为了更

  linqtocsv文件有不太好的地方就是:无法设置标题的行数,默认首行就是标题,这不是很尴尬吗?? ?并不是所有的csv文件严格写的首行是标题,下面全是数据,我接受的任务就是读取很多.csv报表数据,里面就有很多前几行是说明性内容,下面才是标题和数据。为了更好的解决这个问题,自己写吧...

  本博客没有照搬linqtocsv全部源码,保留了主要功能,并对其优化,为我所用,哈哈...

  

  下面是主要代码:

  1-主文件CsvHelper:

  这里在独自解析数据的时候,遇到了很多坑:

  a-遇到数据含有分隔符的问题的解决办法,代码已经包含了

  b-遇到了解析源文档数据时,未指定字符编码时,部分数据丢失导致csv文件个别行数据解析异常的问题,针对该问题,就是老老实实把读取文件时加了字符编码的参数进去,默认UTF-8。  

?

using Microsoft.Extensions.Logging;
 PaymentAccountAPI.Helper;
 System;
 System.Collections.Generic;
 System.IO;
 System.Linq;
 System.Reflection;
 System.Text;

namespace PaymentAccountAPI.CSV
{
    public class CsvHelper
    {
        /// <summary>
        /// 日志
        </summary>
        private ILogger _Logger { get; set; }

        public CsvHelper(ILogger<CsvHelper> logger)
        {
            this._Logger = logger;
        }

        public List<T> Read<T>(string filePath,CsvFileDescription fileDescription) where T : class,new()
        {
            List<T> tList = new List<T>(50 * 10000);

            T t = null;
            int currentRawIndex = 0;

            if (File.Exists(filePath))
            {
                using (StreamReader streamReader =  StreamReader(filePath,fileDescription.Encoding))
                {
                    Dictionary<int,FieldMapper> fieldMapperDic = FieldMapper.GetModelFieldMapper<T>().ToDictionary(m => m.CSVTitleIndex);
                    string rawValue = ;
                    string[] rawValueArray = ;
                    PropertyInfo propertyInfo = string propertyValue = bool rawReadEnd = false;

                    bool isExistSplitChart = do
                    {
                        rawValue = streamReader.ReadLine();

                        //标题行
                        if (currentRawIndex > fileDescription.TitleRawIndex)
                        {
                            if (!string.IsNullOrEmpty(rawValue))
                            {
                                替换字符串含有分隔符为{分隔符},最后再替换回来
                                if (rawValue.Contains("""))
                                {
                                    isExistSplitChart = true;

                                    int yhBeginIndex = ;
                                    int yhEndIndex = string yhText = 
                                    {
                                        yhBeginIndex = StringHelper.GetIndexOfStr(rawValue,",1)">1);
                                        yhEndIndex = StringHelper.GetIndexOfStr(rawValue,1)">2);
                                        yhText = rawValue.Substring(yhBeginIndex,(yhEndIndex - yhBeginIndex + ));
                                        string newYHText = yhText.Replace("").Replace(fileDescription.SeparatorChar.ToString(),1)">{分隔符});
                                        rawValue = rawValue.Replace(yhText,newYHText);
                                    } while (rawValue.Contains());
                                }

                                rawValueArray = rawValue.Split(fileDescription.SeparatorChar);

                                t =  T();
                                foreach (var fieldMapper in fieldMapperDic)
                                {
                                    propertyInfo = fieldMapper.Value.PropertyInfo;
                                    propertyValue = rawValueArray[fieldMapper.Key - ];
                                    .IsNullOrEmpty(propertyValue))
                                    {
                                        try
                                        {
                                            if (isExistSplitChart && propertyValue.Contains())
                                            {
                                                propertyValue = propertyValue.Replace(,fileDescription.SeparatorChar.ToString());
                                            }

                                            TypeHelper.SetPropertyValue(t,propertyInfo.Name,propertyValue);
                                        }
                                        catch (Exception e)
                                        {
                                            this._Logger.LogWarning(e,$第{currentRawIndex + 1}行数据{propertyValue}转换属性{propertyInfo.Name}-{propertyInfo.PropertyType.Name}失败!);
                                            continue;
                                        }
                                    }
                                }
                                tList.Add(t);
                            }
                            else
                            {
                                rawReadEnd = ;
                            }
                        }
                        currentRawIndex++;
                    } while (rawReadEnd == );
                }
            }


            return tList;
        }

        void WriteFile<T>(string path,List<T> tList,1)">()
        {
            .IsNullOrEmpty(path))
            {
                string fileDirectoryPath = ;
                if (path.Contains())
                {
                    fileDirectoryPath = path.Substring(0,path.LastIndexOf(''));
                }
                
                {
                    fileDirectoryPath = path.Substring(/Directory.Exists(fileDirectoryPath))
                {
                    Directory.CreateDirectory(fileDirectoryPath);
                }

                int dataCount = tList.Count;
                Dictionary< m.CSVTitleIndex);
                int titleCount = fieldMapperDic.Keys.Max();
                new [titleCount];
                StringBuilder rawValueBuilder =  StringBuilder();
                ;
                T t = ;
                PropertyInfo propertyInfo = int tIndex = ;

                using (StreamWriter streamWriter = new StreamWriter(path,fileDescription.Encoding))
                {
                    
                    {
                        
                        {
                            rawValue = "";

#if DEBUG
                            if (currentRawIndex % 10000 == )
                            {
                                this._Logger.LogInformation($已写入文件:{path},数据量:{currentRawIndex});
                            }
#endif

                            if (currentRawIndex >= fileDescription.TitleRawIndex)
                            {
                                清空数组数据
                                for (int i = 0; i < titleCount; i++)
                                {
                                    rawValueArray[i] = ;
                                }

                                 fileDescription.TitleRawIndex)
                                {
                                    t = tList[tIndex];
                                    tIndex++;
                                }
                                var fieldMapperItem  fieldMapperDic)
                                {
                                    写入标题行
                                    if (currentRawIndex == fileDescription.TitleRawIndex)
                                    {
                                        rawValueArray[fieldMapperItem.Key - 1] = fieldMapperItem.Value.CSVTitle;
                                    }
                                    真正的数据从标题行下一行开始写
                                    
                                    {
                                        propertyInfo = fieldMapperItem.Value.PropertyInfo;
                                        object propertyValue = propertyInfo.GetValue(t);
                                        string formatValue = ;
                                        if (propertyValue != )
                                        {
                                            if (propertyInfo.PropertyType is IFormattable && !.IsNullOrEmpty(fieldMapperItem.Value.OutputFormat))
                                            {
                                                formatValue = ((IFormattable)propertyValue).ToString(fieldMapperItem.Value.OutputFormat,1)">);
                                            }
                                            
                                            {
                                                formatValue = propertyValue.ToString();
                                            }

                                            如果属性值含有分隔符,则使用双引号包裹
                                             (formatValue.Contains(fileDescription.SeparatorChar.ToString()))
                                            {
                                                formatValue = $"{formatValue}";
                                            }
                                            rawValueArray[fieldMapperItem.Key -  formatValue;
                                        }
                                    }
                                }
                                rawValue = .Join(fileDescription.SeparatorChar,rawValueArray);
                            }
                            rawValueBuilder.Append(rawValue + rn);
                        }
                         (Exception e)
                        {
                            (异常)Excel第{currentRawIndex+1}行,数据列表第{tIndex + 1}个数据写入失败!rawValue:{rawValue});
                            throw;
                        }

                        currentRawIndex++while (tIndex < dataCount);
                    streamWriter.Write(rawValueBuilder.ToString());

                    streamWriter.Close();
                    streamWriter.Dispose();
                }
            }
        }

    }
}

?

  2-CSV映射类特性:

  

 System;

 PaymentAccountAPI.CSV
{
    <summary>
     Csv文件类特性标记
    </summary>
    [System.AttributeUsage(System.AttributeTargets.Field | System.AttributeTargets.Property,AllowMultiple = )]
     CsvColumnAttribute : System.Attribute
    {
        internal const int defaultTitleIndex = Int32.MaxValue;
         标题
        string Title { ; }
         标题位置(从0开始)
        int TitleIndex {  字符输出格式(数字和日期类型需要)
        string OutputFormat { public CsvColumnAttribute()
        {
            Title = ;
            TitleIndex = defaultTitleIndex;
            OutputFormat = ;
        }

        public CsvColumnAttribute(string title,1)">int titleIndex,1)"> outputFormat)
        {
            Title = title;
            TitleIndex = titleIndex;
            OutputFormat = outputFormat;
        }
    }
}

?

  3-CSV文件描述信息类:

  

 CsvFileDescription
    {
        public CsvFileDescription() : this()
        {
        }
        public CsvFileDescription(int titleRawIndex) : char separatorChar,1)">int titleRawIndex,Encoding encoding)
        {
            this.SeparatorChar = separatorChar;
            this.TitleRawIndex = titleRawIndex;
            this.Encoding = encoding;
        }

         CSV文件字符编码
        public Encoding Encoding { ; }

         分隔符(默认为(,),也可以是其他分隔符如(t))
        char SeparatorChar {  标题所在行位置(默认为0,没有标题填-1)
        int TitleRawIndex { ; }

    }
}

?

  4-映射类获取关系帮助类:

  

 System.Reflection;

 字段映射类
    </summary>
     FieldMapper
    {
         属性信息
        public PropertyInfo PropertyInfo { string CSVTitle {  标题下标位置
        int CSVTitleIndex { static List<FieldMapper> GetModelFieldMapper<T>()
        {
            List<FieldMapper> fieldMapperList = new List<FieldMapper>(100);

            List<PropertyInfo> tPropertyInfoList = typeof(T).GetProperties().ToList();
            CsvColumnAttribute csvColumnAttribute = var tPropertyInfo  tPropertyInfoList)
            {
                csvColumnAttribute = (CsvColumnAttribute)tPropertyInfo.GetCustomAttribute((CsvColumnAttribute));
                if (csvColumnAttribute != )
                {
                    fieldMapperList.Add( FieldMapper
                    {
                        PropertyInfo = tPropertyInfo,CSVTitle = csvColumnAttribute.Title,CSVTitleIndex = csvColumnAttribute.TitleIndex,OutputFormat = csvColumnAttribute.OutputFormat
                    });
                }
            }
             fieldMapperList;
        }

    }

}

?

  5-其他扩展类:

  

 PaymentAccountAPI.Helper
{
     StringHelper
    {
         获取字符串中第strPosition个位置的str的下标
        </summary>
        <param name="text"></param>
        <param name="str"></param>
        <param name="strPosition"></param>
        <returns></returns>
        static int GetIndexOfStr(string text,1)">string str,1)"> strPosition)
        {
            int strIndex = -int currentPosition = string.IsNullOrEmpty(text) && !string.IsNullOrEmpty(str) && strPosition >= )
            {
                
                {
                    currentPosition++if (strIndex == -)
                    {
                        strIndex = text.IndexOf(str);
                    }
                    
                    {
                        strIndex = text.IndexOf(str,strIndex + );
                    }
                } while (currentPosition < strPosition);
            }

             strIndex;
        }
    }
}

?

?

  最后就是将CsvHelper注入到单例中,就可以使用了...

(编辑:李大同)

【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容!

    推荐文章
      热点阅读