ruby – 解析巨大的(~100mb)kml(xml)文件,花费*小时*,没有任何实

解析代码很好,事实上我只是为了它的分享它,即使这段代码与我的问题没有太大关系：

geofactory = RGeo::Geographic.projected_factory(:projection_proj4 => "+proj=lcc +lat_1=34.83333333333334 +lat_2=32.5 +lat_0=31.83333333333333 +lon_0=-81 +x_0=609600 +y_0=0 +ellps=GRS80 +to_meter=0.3048 +no_defs",:projection_srid => 3361)
f = File.open("horry_parcels.kml")
kmldoc = Nokogiri::XML(f)

kmldoc.css("//Placemark").each_with_index do |placemark,i|
      puts i
      tds = Nokogiri::HTML(placemark.search("//description").children[0].to_html).search("tr > td")
      h = HorryParcel.new
      h.owner_name = tds.shift.text
      tds.shift
      tds.each_slice(2) do |k,v|
        col = k.text.downcase
        eval("h.#{col} = v.text")
      end
      coords = kmldoc.search("//MultiGeometry")[i].text.gsub("n","").gsub("t","").split(",0 ").map {|x| x.split(",")}
      points = coords.map { |lon,lat| geofactory.parse_wkt("POINT (#{lon} #{lat})") }
      geo_shape = geofactory.polygon(geofactory.linear_ring(points))
      proj_shape = geo_shape.projection
      h.geo_shape = geo_shape
      h.proj_shape = proj_shape
      h.save
    end

无论如何,我已经用更小,更小的kml样本测试了这段代码并且它有效.

然而,当我加载真实的东西时,ruby只是等待,好像它正在处理一些东西.然而,这种“处理”现在已经持续了几个小时,而我一直在做其他事情.您可能已经注意到,我在地标数组上有一个计数器(each_with_index),在这个多小时内,没有一个i值被放到命令行中.奇怪的是它还没有超时,但即使这样有效,也必须有更好的方法来做这件事.

我知道我可以在谷歌地球(这里是Google Earth Pro)中打开KML文件,并将数据保存在更小,更易于管理的kml文件中,但是看起来设置的方式,这将是一个非常手动,不专业的过程.

这是kml的样本(只有一个地标),如果这有帮助的话.

编辑：
我使用的99.9％的数据是* .shp格式,所以我在过去一周内忽略了这个问题.但是我要在我的桌面计算机上运行这个过程(从我的笔记本电脑上运行)并运行它直到它超时或完成.


上面的代码应该告诉我需要多长时间.从那里(如果它确实完成)我们应该能够计算一个粗略的经验法则,你应该期望你的(否则完美的)代码在没有SAX解析或文档组件的“雾化”的情况下运行多长时间.
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2" xmlns:kml="http://www.opengis.net/kml/2.2" xmlns:atom="http://www.w3.org/2005/Atom">
<Document>
    <name>justone.kml</name>
    <Style id="PolyStyle00">
        <LabelStyle>
        <color>00000000</color>
        <scale>0</scale>
    </LabelStyle>
    <LineStyle>
        <color>ff0000ff</color>
    </LineStyle>
    <PolyStyle>
        <color>00f0f0f0</color>
    </PolyStyle>
</Style>
<Folder>
    <name>justone</name>
    <open>1</open>
    <Placemark id="ID_010161">
        <name>STUART CHARLES A JR</name>
        <Snippet maxLines="0"></Snippet>
        <description>""</description>
        <styleUrl>#PolyStyle00</styleUrl>
        <MultiGeometry>
            <Polygon>
                <outerBoundaryIs>
                    <LinearRing>
                        <coordinates>
                            -78.941896,33.867893,0     -78.942514,33.868632,0 -78.94342899999999,33.869705,0 -78.943708,33.870083,0 -78.94466799999999,33.871142,0 -78.94511900000001,33.871639,0 -78.94541099999999,33.871776,0 -78.94635,33.872216,0 -78.94637899999999,33.872229,0 -78.94691400000001,33.87248,0 -78.94708300000001,33.87256,0 -78.94783700000001,33.872918,0 -78.947889,33.872942,0 -78.948655,33.873309,0 -78.949589,33.873756,0 -78.950164,33.87403,0 -78.9507,33.873432,0 -78.95077000000001,33.873384,0 -78.950867,33.873354,0 -78.95093199999999,33.873334,0 -78.952518,33.871631,0 -78.9540
geofactory = RGeo::Geographic.projected_factory(:projection_proj4 => "+proj=lcc +lat_1=34.83333333333334 +lat_2=32.5 +lat_0=31.83333333333333 +lon_0=-81 +x_0=609600 +y_0=0 +ellps=GRS80 +to_meter=0.3048 +no_defs",lat| geofactory.parse_wkt("POINT (#{lon} #{lat})") }
      geo_shape = geofactory.polygon(geofactory.linear_ring(points))
      proj_shape = geo_shape.projection
      h.geo_shape = geo_shape
      h.proj_shape = proj_shape
      h.save
    end
geofactory = RGeo::Geographic.projected_factory(:projection_proj4 => "+proj=lcc +lat_1=34.83333333333334 +lat_2=32.5 +lat_0=31.83333333333333 +lon_0=-81 +x_0=609600 +y_0=0 +ellps=GRS80 +to_meter=0.3048 +no_defs",lat| geofactory.parse_wkt("POINT (#{lon} #{lat})") }
      geo_shape = geofactory.polygon(geofactory.linear_ring(points))
      proj_shape = geo_shape.projection
      h.geo_shape = geo_shape
      h.proj_shape = proj_shape
      h.save
    end00001,33.869583,0 -78.955254,33.867865,0 -78.954606,33.867499,0 -78.953833,33.867172,0 -78.952994,33.866809,0 -78.95272799999999,33.867129,0 -78.952139,33.866803,0 -78.95152299999999,33.86645,0 -78.95134299999999,33.866649,0 -78.95116400000001,33.866847,0 -78.949281,33.867363,0 -78.948936,33.866599,0 -78.94721699999999,33.866927,0 -78.941896,0
 </coordinates>
 </LinearRing>
 </outerBoundaryIs>
 </Polygon>
 </MultiGeometry>
 </Placemark>
 </Folder>
 </Document>
 </kml>
class ClassName

attr_reader :before,:after

def go
  @before = Time.now
  run_actual_code
  @after = Time.now
  puts "process took #{(@after - @before) seconds} to complete"
end

def run_actual_code
  ...
end

end