top 50 comments

sorted by: hot top controversial new old
[–] 293 points 2 years ago (4 children)

Maybe it's time we invent JPUs (json processing units) to equalize the playing field.

  • source
  • hideshow 7 child comments
  • [–] [S] 221 points 2 years ago (3 children)

    The best I can do is an ML model running on an NPU that parses JSON in subtly wrong and impossible to debug ways

  • source
  • parent
  • hideshow 5 child comments
  • [–] 58 points 2 years ago (1 child)

    Just make it a LJM (Large JSON Model) capable of predicting the next JSON token from the previous JSON tokens and you would have massive savings in file storagre and network traffic from not having to store and transmit full JSON documents all in exchange for an "acceptable" error rate.

  • source
  • parent
  • hideshow 2 child comments
  • load more comments (1 reply)
  • load more comments (1 reply)
    [–] 165 points 2 years ago (4 children)

    Well, do you have dedicated JSON hardware?

  • source
  • hideshow 7 child comments
  • [–] 105 points 2 years ago (2 children)

    Please no, don't subsidize anything Java-Script. It will only make it less efficient.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 26 points 2 years ago (3 children)

    My thoughts on software in general over the past 20 years. So many programs inefficiently written and in 4th level languages just eats up any CPU/memory gain. (Less soap box and more of a curious what if to how fast things would be if we still wrote highly optimized programs)

  • source
  • parent
  • hideshow 5 child comments
  • [–] 36 points 2 years ago*

    Answer: there'd be far less software in the world, it would all be more archaic and less useful, and our phones and laptops would just sit at 2% utilization most of the time.

    There's an opportunity cost to everything, including fussing over whether that value can be stored as an int instead of a double to save 8 bits of space. High level languages let developers express their feature and business logic faster, with fewer bugs, and much lower ongoing maintenance costs.

  • source
  • parent
  • load more comments (1 reply)
  • [–] 24 points 2 years ago (7 children)
  • [–] 24 points 2 years ago (3 children)

    The R in ARM and RISC is a lie.

  • source
  • parent
  • hideshow 4 child comments
  • load more comments (2 replies)
  • load more comments (6 replies)
  • load more comments (1 reply)
    [–] 120 points 2 years ago (5 children)

    Everybody gangsta still we invent hardware accelerated JSON parsing

  • source
  • hideshow 6 child comments
  • [–] 98 points 2 years ago (2 children)

    https://ieeexplore.ieee.org/document/9912040 "Hardware Accelerator for JSON Parsing, Querying and Schema Validation" "we can parse and query JSON data at 106 Gbps"

  • source
  • parent
  • hideshow 4 child comments
  • [–] 26 points 2 years ago (2 children)

    106 Gbps

    They get to this result on 0.6 MB of data (paper, page 5)

    They even say:

    Moreover, there is no need to evaluate our design with datasets larger than the ones we have used; we achieve steady state performance with our datasets

    This requires an explanation. I do see the need - if you promise 100Gbps you need to process at least a few Tbs.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 13 points 2 years ago (3 children)

    Imagine you have a car powered by a nuclear reactor with enough fuel to last 100 years and a stable output of energy. Then you put it on a 5 mile road that is comprised of the same 250 small segments in various configurations, but you know for a fact that starts and ends at the same elevation. You also know that this car gains exactly as much performance going downhill as it loses going uphill.

    You set the car driving and determine that, it takes 15 minutes to travel 5 miles. You reconfigure the road, same rules, and do it again. Same result, 15 minutes. You do this again and again and again and always get 15 minutes.

    Do you need to test the car on a 20 mile road of the same configuration to know that it goes 20mph?

    JSON is a text-based, uncompressed format. It has very strict rules and a limited number of data types and structures. Further, it cannot contain computational logic on it's own. The contents can interpreted after being read to extract logic, but the JSON itself cannot change it's own computational complexity. As such, it's simple to express every possible form and complexity a JSON object can take within just 0.6 MB of data. And once they know they can process that file in however-the-fuck-many microseconds, they can extrapolate to Gbps from there

  • source
  • parent
  • hideshow 3 child comments
  • load more comments (3 replies)
  • load more comments (1 reply)
  • load more comments (4 replies)
    [–] 103 points 2 years ago (3 children)

    Render the json as polygons?

  • source
  • hideshow 6 child comments
  • [+] 91 points 2 years ago* (last edited 2 years ago) (3 children)
  • load more comments (2 replies)
    [–] 76 points 2 years ago (1 child)

    That is sometime the issue when your code editor is a disguised web browser 😅

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 55 points 2 years ago* (1 child)

    there are simd accelerated json decoders

  • source
  • hideshow 2 child comments
  • [–] 43 points 2 years ago (1 child)

    CPU vs GPU tasks I suppose.

  • source
  • hideshow 2 child comments
  • [–] 43 points 2 years ago (3 children)

    Would you rather have 100,000 kg of tasty supreme pizza, or 200 kg of steaming manure?

    Choose wisely.

  • source
  • hideshow 4 child comments
  • load more comments (2 replies)
    [–] 35 points 2 years ago (4 children)

    I have the same problem with XML too. Notepad++ has a plugin that can format a 50MB XML file in a few seconds. But my current client won't allow plugins installed. So I have to use VS Code, which chokes on anything bigger than what I could do myself manually if I was determined.

  • source
  • hideshow 5 child comments
  • [–] [S] 12 points 2 years ago (1 child)

    Time to train an LLM to format XML and hope for the best

  • source
  • parent
  • hideshow 2 child comments
  • load more comments (3 replies)
    [–] 25 points 2 years ago

    Someone just needs to make a GPU-accelerated JSON decoder

  • source
  • [–] 24 points 2 years ago (1 child)
  • [–] 16 points 2 years ago (15 children)
  • load more comments (14 replies)
    load more comments
    view more: next ›