i would probably use Playwright with custom code, create chunks based on similar products, then run it on a large cluster in parallel (https://github.com/Burla-Cloud/burla).
if you have a single worker trying to scrape a shit ton of products back to back to back you're going to get rate limited or their bot detection will catch you.
do we still need unreal engine and unity? if yes what are the things that raylib is missing that these engines have? beginner gamedev so please take it easy here
Those engines serve a different purpose than a library like Raylib. They give you a bunch of stuff out of the box like lighting, raytracing (esp Unreal), pathfinding, and a ton of helper functions used in making a game like vector calculations.
Raylib helps you draw stuff, play sound, and do the basics. But you're gonna be writing your own lighting/raytrace/pathfind/etc functions and it's ultimately going to take longer. The upside is if you need to do something very unique, all of the power to mske it reality is in your hands because Raylib isn't opinionated on how your game logic works and how it's packaged up. It's just the delivery guy to give the result to the user.
I think engines like Unreal, Unity, and Godot will remain popular with people who are more interested in creating a game than creating a game engine.
> if yes what are the things that raylib is missing that these engines have?
Asset management and import pipelines, rendering pipeline, loads of ready-to-go features like environments, baked lighting and global illumination, AO, reflections, particle systems, input mapping and event propagation, scripting, audio systems, GUI systems, and lots more.
Raylib is a library that you could use to build all that stuff, but otherwise it's a useful library, not a fully-featured game engine.
If you don't want a game engine and know exactly what you features you need and want to build only that, then Raylib is a great option.
If you don't want to write a global illumination system or asset management pipeline but would rather focus on creating gameplay, then a game engine is a good choice.
Raylib is more of a low level runtime library than an engine. Godot, Unity, Unreal etc. come with very extensive interactive tooling for game creation. Modern engines are really about interactive content editing and collaboration in the development process. This is essentially table stakes for game development in larger teams, and comes with a lot of added internal complexity.
Raylib is for hobbyists that want control over everything, but don't want to go through the hassle of dealing with DirectX/OpenGL. It isn't competing with Unreal/Unity at all.
I don't know that any open source project will ever compete with Unreal and its high end tech targeting the AAA space. But beyond that, Raylib doesn't give a ton of things an engine does because that is not the point of a library like it. This is to let someone build their own engine how they want, Unreal/Unity/Godot/etc let you give up some control and decision options to skip a ton of work building out a lot of basic features.
The scope/feature set of both is just quite a lot wider, from IDEs to an ecosystem of 1st and 3rd party libraries and extensions. The rendering engines and their capabilities are also quite different (with Unreal and Unity both being quite a lot more advanced).
The only thing that might save civilisation at this point is some kind of magic carbon capture technology.
Or a giant sun shade.
Or perhaps planetary-scale reforesting. But that's more speculative.
We passed the "we need to reduce emissions" marker at least a decade ago.
Emissions don't just need to be reduced, they need to be objectively greenhouse gas negative for a good long while. Meaning centuries.
If that doesn't happen - and it's looking really unlikely - catastrophic change is locked in.
The result will be population destruction at a scale never seen in all of recorded history. That's going to do more to minimise emissions than veganism will.
There are many projects which capture carbon and produces fuel, including growing crops, they are just more expensive than oil. So, realistic scenario at this point is some war which will destroy oil industry in certain areas (gulf, Russia), and carbon capture will be more viable.
There are some 50 story buildings. But cost increases pretty dramatically. Especially when you build on fill and mess up the foundation work [1]. There's a fairly consistent stream of tall buildings built over the years, not all of them are 50+ floors though [2]. There's gaps in building during economic distress, as you would expect.
Because of the expense, high rises are either offices or luxury homes. It doesn't make sense to build a 50 story flop house. Zoning also pushes you to build high rises in the neighborhood of other high rises, and land is expensive there. All in all, better to build a 6-12 story midrise flophouse in a cheaper part of town.
Also, from looking at the street view, and what a listing site said, it's pretty clear this particular 2 story building is pretty old (1919), and the neighboring buildings have been built up, but this one held out. Chances are, one day, a new, larger building will replace or subsume it. That's the circle of life, as happened to its neighbors.
I'm not sure about the cost. A lot of Mumbai's middle class housing is 30-40 floor towers. These are flats that sell for well under 200k, most under 100k. Labor cost is lower in India, but most material and equipment costs are not.
Another cost difference that you're not accounting for is building code. It would be entirely unacceptable for an SF skyscraper to fall down in a 7.9 magnitude earthquake. Likewise, fire safety is a huge factor in the US, and especially in California. Safe evacuation routes for people get more and more difficult to provide when the number of stories climbs higher, too.
The reveal that the pods building is only 2 stories is pretty funny to me. 12 Mint Plaza btw for anyone who wants to poke around the area in street-view. It's right next to an 8 story apartment building, across the block from what must be a 20+ one, so it's not like tall buildings are infeasible there.
Some folks have actively fought against the "Manhattan-ification" of SF for many years. So much so they have a name for them "NIMBY" standing for not in my backyard. Now there are YIMBY folks too. It's a whole thing.
Because we mustn't disturb the unique character of the box-like 2-storey homes stretching off into the Sunset district (and served by 2 different light rail lines, no less).
Because while generations have passed since 1906, building codes have not forgot the fate of most of San Francisco's buildings, and the fault lines that most all Californians deal with.
- for whatever reason, even the same sized directory takes much much longer than its corresponding archive version when using this tool
reply