Home Page > News List > Cloud Tech >> Cloud Tech

Nani? The backend needs to return me 100000 pieces of data at once! Let's take a look at my eight clever solutions

Cloud Tech Source: Network

10what the f**k!.


  • 10
  • what the f**k!

...

  • ......

express

expressVue+Express+Mysql

jsroute.get("/bigData", (req, res) => {  res.header('Access-Control-Allow-Origin', '*'); //   let arr = [] //   for (let i = 0; i < 100000; i++) { //     arr.push({      id: i + 1,      name: '' + (i + 1),      value: i + 1,    })  }  res.send({ code: 0, msg: '', data: arr }) // })

html

html<el-button :loading="loading" @click="plan"></el-button><el-table :data="arr">  <el-table-column type="index" label="" />  <el-table-column prop="id" label="ID" />  <el-table-column prop="name" label="" />  <el-table-column prop="value" label="" /></el-table>data() {    return {      arr: [],      loading: false,    };},async plan() {    // arr}

10

js async plan() {      this.loading = true;      const res = await axios.get("http://ashuai.work:10000/bigData");      this.arr = res.data.data;      this.loading = false;}

  • 210
  • 1010
  • 10

  • 10
  • 100~910~19
  • [1,2,3,4,5,6,7]
  • 3
  • [ [1,2,3], [4,5,6], [7]]

10

jsfunction averageFn(arr) {  let i = 0; // 1. 0  let result = []; // 2.   while (i < arr.length) { // 6.     // 3.     result.push(arr.slice(i, i + 10)); // 4. 10    i = i + 10; // 5. 10  }  return result; // 7. }

js  async plan() {      this.loading = true;      const res = await axios.get("http://ashuai.work:10000/bigData");      this.loading = false;      let twoDArr = averageFn(res.data.data);      for (let i = 0; i < twoDArr.length; i++) {        //         setTimeout(() => {          this.arr = [...this.arr, ...twoDArr[i]]; //         }, 1000 * i); // 17 * i // ... 17 = 1000 / 60      }    },

requestAnimationFrame

requestAnimationFramerequestAnimationFrame

plan

jsasync plan() {  this.loading = true;  const res = await axios.get("http://ashuai.work:10000/bigData");  this.loading = false;  // 1.   let twoDArr = averageFn(res.data.data);  // 2.   const use2DArrItem = (page) => {    // 4.     if (page > twoDArr.length - 1) {      console.log("");      return;    }    // 5.     requestAnimationFrame(() => {      // 6. concat      this.arr = [...this.arr, ...twoDArr[page]];      // 7.       page = page + 1;      // 8.       use2DArrItem(page);    });  };  // 3. 0  use2DArrItem(0); },

jsgetShowTableData() {     //      let begin = (this.pageIndex - 1) * this.pageSize;     //      let end = this.pageIndex * this.pageSize;     //     this.showTableData = this.allTableData.slice(begin, end); }

500MB5010MB......

  • scrollTop + clientHeight >= innerHeight
  • new MutationObserver()


v-el-table-infinite-scrollnpmwww.npmjs.com/package/el-

vue-scroller

el-tableel-table-infinite-scroll

vue2vue3

cnpm install --save [email protected]

js// import elTableInfiniteScroll from 'el-table-infinite-scroll';Vue.use(elTableInfiniteScroll);

el-table

js<el-table  v-el-table-infinite-scroll="load"  :data="tableData">  <el-table-column prop="id" label="ID"></el-table-column>  <el-table-column prop="name" label=""></el-table-column></el-table>async load() {    // ...},

html<template>  <div class="box">    <el-table      v-el-table-infinite-scroll="load"      height="600"      :data="tableData"      border      style="width: 80%"      v-loading="loading"      element-loading-text="..."      element-loading-spinner="el-icon-loading"      element-loading-background="rgba(255, 255, 255, 0.5)"      :header-cell-style="{        height: '24px',        lineHeight: '24px',        color: '#606266',        background: '#F5F5F5',        fontWeight: 'bold',      }"    >      <el-table-column type="index" label=""></el-table-column>      <el-table-column prop="id" label="ID"></el-table-column>      <el-table-column prop="name" label=""></el-table-column>      <el-table-column prop="value" label=""></el-table-column>    </el-table>  </div></template><script>// function averageFn(arr) {  let i = 0;  let result = [];  while (i < arr.length) {    result.push(arr.slice(i, i + 10)); // 10    i = i + 10; // 1010  }  return result;}import axios from "axios";export default {  data() {    return {      allTableData: [], //       tableData: [], //       loading: false    };  },  //   async created() {    this.loading = true;    const res = await axios.get("http://ashuai.work:10000/bigData");    this.allTableData = averageFn(res.data.data); //     // this.originalAllTableData = this.allTableData //     this.loading = false;    //     this.load();   },  methods: {    //     async load() {      console.log("");      // return      if (this.allTableData.length == 0) {        console.log("");        return;      }      //       let arr = this.allTableData[0];      this.tableData = this.tableData.concat(arr);      //       this.allTableData.shift();    },  },};</script>

/

  • 10dom
  • dom

GitHub

html<template>  <!--             104010400            400 -->  <div    class="virtualListWrap"    ref="virtualListWrap"    @scroll="handleScroll"    :style="{ height: itemHeight * count + 'px' }"  >    <!-- dom -->    <div      class="placeholderDom"      :style="{ height: allListData.length * itemHeight + 'px' }"    ></div>    <!-- 10top -->    <div class="contentList" :style="{ top: topVal }">      <!--  -->      <div        v-for="(item, index) in showListData"        :key="index"        class="itemClass"        :style="{ height: itemHeight + 'px' }"      >        {{ item.name }}      </div>    </div>    <!--  -->    <div class="loadingBox" v-show="loading">      <i class="el-icon-loading"></i>        <span>loading...</span>    </div>  </div></template><script>import axios from "axios";export default {  data() {    return {      allListData: [], //       itemHeight: 40, // 40      count: 10, //       start: 0, //       end: 10, //       topVal: 0, // top      loading: false,    };  },  computed: {    // allListDatashowListData    showListData: function () {      return this.allListData.slice(this.start, this.end);    },  },  async created() {    this.loading = true;    const res = await axios.get("http://ashuai.work:10000/bigData");    this.allListData = res.data.data;    this.loading = false;  },  methods: {    //     handleScroll() {      /**       * Element.scrollTop       *       *        * 40.84/0.8 = 5       *       * 10       * ,  + size==,        * */      const scrollTop = this.$refs.virtualListWrap.scrollTop;      this.start = Math.floor(scrollTop / this.itemHeight);      this.end = this.start + this.count;      /**       * top       * */      this.topVal = this.$refs.virtualListWrap.scrollTop + "px";    },  },};</script><style scoped lang="less">// .virtualListWrap {  box-sizing: border-box;  width: 240px;  border: solid 1px #000000;  //   overflow-y: auto;  //   position: relative;  .contentList {    width: 100%;    height: auto;    //     position: absolute;    top: 0;    left: 0;    .itemClass {      box-sizing: border-box;      width: 100%;      height: 40px;      line-height: 40px;      text-align: center;    }    //     .itemClass:nth-child(even) {      background: #c7edcc;    }    .itemClass:nth-child(odd) {      background: pink;    }  }  .loadingBox {    position: absolute;    top: 0;    left: 0;    right: 0;    bottom: 0;    width: 100%;    height: 100%;    background-color: rgba(255, 255, 255, 0.64);    color: green;    display: flex;    justify-content: center;    align-items: center;  }}</style>

vxetable

tableUIvxetable

vue2vue3 5w 30w

vxetable.cn/v3/#/table/

cnpm i xe-utils [email protected] --save

main.js

js// VXETableimport VXETable from 'vxe-table'import 'vxe-table/lib/style.css'Vue.use(VXETable)

html<template>  <div class="box">    <vxe-table      border      show-overflow      ref="xTable1"      height="300"      :row-config="{ isHover: true }"      :loading="loading"    >      <vxe-column type="seq"></vxe-column>      <vxe-column field="id" title="ID"></vxe-column>      <vxe-column field="name" title=""></vxe-column>      <vxe-column field="value" title=""></vxe-column>    </vxe-table>  </div></template><script>import axios from "axios";export default {  data() {    return {      loading: false,    };  },  async created() {    this.loading = true;    const res = await axios.get("http://ashuai.work:10000/bigData");    this.loading = false;    this.render(res.data.data);  },  methods: {    render(data) {      this.$nextTick(() => {        const $table = this.$refs.xTable1;        $table.loadData(data);      });    },  },};</script>

Web Worker

Web Worker

Web Workervue-workerWeb Worker

...

7

10

7

1010

****sqllimitdom

...

...

UI

2200007...

...

lighthousechromedevtoolPerformance

4000lighthousechormePerformance

lighthouse

lighthouseWeb

19

lighthouse

lighthouse

lighthousejs

  • 9.4s
    • 14
    • js7.2s
  • dom
  • 24530domdommarkerdomdomdom
  • 48kb1.3s

lighthouse

  • 14
  • dom
  • 1.4s

lighthousechrome devtoolPerformance

Performance

PerformancePerformancemain

  • htmljscomposite layers
  • js

chrome Performance

10sjs7.6sjsMicrotasks3.4s_next3.94sfulfilled

  • _nextmarker

  • fulfilled_nextmarkerpopupdommarkerpopuphtml

fulfilledmarkerpopup

lighthousePerformance:

  • markerdomdom
  • marker_next

  • dommarkercanvas
  • marker

n

diffdemo

code.juejin.cn/pen/7142775

  • dom
  • requestAnimationFrame10

DOMdom

  • React

antddemo

4000

_next1sjs4621ms

markerdomdomleafletcanvascanvasdomHTMLCanvas]

markermarkerpopupleafletplugins Leaflet.Canvas-Markersmarkermarkerdemo

marker

jsx<MapContainer {...props}>   {       stops.map((stop) => {         return <CustomMarker key={stop.id} data={stop} />;        })    } </MapContainer>  // CustomMarker function CustomMarker(props) {     ...     // react-leafletMarkerDom     <Marker      ...      position={position}    >      <Popup        {...props}      >        ... // popup      </Popup>    </Marker> }

jsx // render <MapContainer {...props}>   <CanvasMarkers data={stops} /> </MapContainer>  // CanvasMarkers import 'leaflet-canvas-marker'; function CanvasMarkers({data}) {        const map = useMap();      const canvasLayerRef = useRef();      const icon = window.L.icon({      iconUrl: '',      iconSize: [8, 8],      iconAnchor: [4, 4],    });      useEffect(() => {        if (!data.length) {          canvasLayerRef.current?.onRemove(map);          return;        }        canvasLayerRef.current = window.L.canvasIconLayer({}).addTo(map);        const canvasMarkers = [];        for (let i = 0, len = data.length; i < len; i++) {          const { lat, lng } = data[i];          const marker = window.L.marker([lat, lng], {            icon,          }).bindPopup(popupHtml);          canvasMarkers.push(marker);        }        canvasLayerRef.current?.addLayers(canvasMarkers);      }, [markers, icon, map]);      return null; }

dommarkercanvas

4000canvas

fulfilled100msjs1102mscanvas

4000Performancejs7.4s75%

  • 1
  • jsx
  • // table setTableData(tableData); // setTableDatareactuseState setTimeout(() => { // 100ms setMarkers(stops); // setMarkersreactuseState }, 100);
  • 2
  • CanvasMarkers
  • jsx
import 'leaflet-canvas-marker'; function CanvasMarkers({data}) { ... const [data, setData] = useState([]); // marker const sliceData = useCallback((list, index: number = 0, num = 100) => { const endIdx = Math.ceil(list.length / num); if (index === endIdx) { return; } setTimeout(() => { // 200ms const toBeRenderList = list.slice(index * num, (index + 1) * num); setData(toBeRenderList); console.log(toBeRenderList, toBeRenderList.length); sliceData(list, index + 1, num); }, 200); }, []); useEffect(() => { const len = markers.length; if (len === 0) { canvasLayerRef.current?.onRemove(map); return; } canvasLayerRef.current = window.L.canvasIconLayer({}).addTo(map); //  const sliceLen = len > 100000 ? 2000 : len > 50000 ? 1000 : 500; sliceData(data, 0, sliceLen); }, [markers, map, sliceData]); // dataL.marker useEffect(() => { if (!data.length) { return; } const canvasMarkers = []; for (let i = 0, len = data.length; i < len; i++) { const { lat, lng } = data[i]; const marker = window.L.marker([lat, lng], { icon, }).bindPopup(popupHtml); canvasMarkers.push(marker); } canvasLayerRef.current?.addLayers(canvasMarkers); }, [data, map]); return null; }

5Performance

TBT: 2690ms1045ms60% 4.5s6.1s1.6s

4000

  • js7453ms1466ms
  • 9979ms2999ms

  • ashuai.work:8888/#/bigData
  • GitHubgithub.com/shuirongshu


Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])

at Nani The backend needs to return me 100000

Guess you like