Acerca de
I have more than 10 years of experience in Data Engineering, Data Warehousing, and Data…
Actividad
1 mil seguidores
-
Pavel Zhiriakov ha compartido estoPavel Zhiriakov ha compartido estoHere is McDonald's UK expansion strategy 2023 → 2026. Like, share, ping me to get the same for other brands! McDonald's’ didn’t expand randomly. They doubled down where they were already strong and slowed down where everyone thinks the biggest opportunity is. 📊 #UK, 2023 → 2026: • South East → biggest absolute growth (+24 stores) • North West → already #1 density still expanding • London → almost no growth (near saturation) • Northern Ireland → +22% density (highest relative gain) Even with publicly available data, you can reverse-engineer expansion strategy. Now imagine what you can do with real delivery & demand data. Built with Getplace.io (yes, this is exactly what we do for our clients)
-
Pavel Zhiriakov ha compartido estoWow!Pavel Zhiriakov ha compartido estoTogether with NVIDIA RAPIDS we bring GPU acceleration to Polars! Read more in our release blog: https://lnkd.in/e2HNKBRK
-
Pavel Zhiriakov ha publicado estoRDD’s in Spark are not quite immutable. That’s what my Spark mentor told me while ago. Whole this week I’ve tried to remember - Why? Note: “Before Spark 2.0.0, we had to perform transformations directly on RDDs, using a lot of external functions. And now we have a cool abstraction as dataframes. So under the hood, we still have the same old RDDs” Let’s make a quick recap. In Spark - RDD stands for Resilient Distributed Dataset. The word resilient means - that Spark is able to recover an RDD from failures, and Spark ensures that the pipeline can continue even if something went wrong. You can imagine RDDs as a DAG (Direct Acyclic Graph), where each RDD object is a node in a compute graph. Every time you apply a transformation in Spark, new RDD is created. Because of the Spark lazy evaluation, the data processing is happening only after we call for some action. Like - count for example. Before that, we are constructing a DAG. Can you guess what going to happen, if some failure going to happen in the moment of loading data to the executor? Correct. Spark will try to restore it, and load again. Basically, if the data will change in the data source at this moment, we might have a problem. A big problem. Generally speaking, RDDs are immutable, because they are just nodes in compute graph, and immutability is applied to a logical plan. The data behind them might be changed, so they are not quite immutable. Your thoughts?💭
-
Pavel Zhiriakov ha compartido estoJust published a new Medium article about Apache Spark! 🚀 I'm breaking down how it works under the hood, showing the principles behind this framework. Let me know what you think!
-
Pavel Zhiriakov ha compartido estoPavel Zhiriakov ha compartido estoIs AI a disruption in the website industry? Yes it is, and we are such great believers in the technology, that we launched our first AI website creator back in 2016, Wix ADI (Artificial Design Intelligence). All the user had to do was write a few details about their business, and boom, in a few minutes, AI created a beautiful and functional website. Today Wix ADI technology powers the onboarding flow for our new Wix Editor. I have been asked so many questions about AI lately, so I feel the urge to share some of my thoughts (not written by AI, I swear). AI will make Wix better. As AI makes it easier for our users to build websites that they love, the value of our integrated platform becomes even bigger. Our biggest challenge is to help our users finish a website that represents the uniqueness of their business. Most of our customers don’t want to spend hours writing text, designing elements, or paying for a professional photographer. ADI has proven it, the faster and easier the website creation is, the happier our users are. AI is a natural extension of what we do, and the better AI algorithms get, the better Wix will become. Am I worried that AI is going to take over website building? The short answer is no. AI has tremendous value in website creation. But it does not substitute the software needed to manage someone’s entire business online. The Wix platform has evolved into a comprehensive business management solution that not only allows our users to design their website, but also manage their entire business online. So can’t AI do all that? A website is a lot more than just text and images. A website is an integrated software platform with millions of lines of code, functionalities like eCommerce, appointments, selling tickets, blogs, and so much more. On top of that, everything has to work together - the databases, CDN, the transactions, privacy and of course complex security mechanisms. What is our AI team doing now? We were pioneers in the industry when we launched ADI, and since then we have grown and improved our AI features, including our most recent, AI Text Creator, that combines our knowledge of website content with ChatGPT. Today, many of our developers are working on AI related products, to empower that, we have an amazing AI department with excellent talent and proven track record. Here are some cool AI-generated tools we are working on: The new generation of ADI - which I personally find groundbreaking Using Large Language Models (LLM), adopted to the Wix language, to help our users get the best solutions tailor-made to their needs. A completely new way of working with media, where AI edits and enhances the images. An AI assistant to help users in building their websites, trained by data from hundreds of millions of websites. These are just a few examples of what's in our pipeline, obviously we won't provide the details of our less obvious roadmap, but stay tuned, we have some incredible things coming up.Wix’s new ADI tool uses artificial intelligence to build a Website for youWix’s new ADI tool uses artificial intelligence to build a Website for you
-
Pavel Zhiriakov ha recomendado estoPavel Zhiriakov ha recomendado estoGartner® just released their new Magic Quadrant™ for Search and Product Discovery today. I can't share the MQ grid itself until Thursday (though if you're a Gartner client, you can already access it in their portal). What I can tell you is that we at Constructor are very proud of the results. Watch this space Thursday when we'll be allowed to post an image of the MQ itself and give you complimentary access to the report. I'm really excited for you all to see it.
-
Pavel Zhiriakov ha recomendado estoPavel Zhiriakov ha recomendado estoOur new book about Databricks on Azure is going live. Stay tuned - more details on availability coming soon! 🚀 #DataEngineering #Databricks #Azure #BigData #Lakeflow #SparkDeclarativePipelines #UnityCatalog #Packt
-
Pavel Zhiriakov ha recomendado estoPavel Zhiriakov ha recomendado estoFellow startup founders and SMB owners: I'm sure you also hate jumping between QuickBooks, Stripe, and HubSpot just to understand your company's finances. What is your runway? Do you have a risk of cash crunch due to an upcoming annual payment? Can you afford to hire? Revisor is a free open-source Claude Cowork plugin that connects to your data sources and immediately tells you what needs attention: - real runway with best/base/worst scenarios, not spreadsheet guesses - warns you before big annual payments hit - weekly briefing every Monday on autopilot No dashboards, no spreadsheet chaos. Just ask questions about your business in plain English. Source: https://lnkd.in/ggNUF_Sy Plugin: https://lnkd.in/gzWhM6JU
-
Pavel Zhiriakov ha recomendado estoThat’s my brother – and I’m lucky to have a front-row seat.Pavel Zhiriakov ha recomendado estoSeven months ago, Dan Shiebler and I started Artemis Security. Today, we’re coming out of stealth with $70M in funding. The round was led by Felicis with First Round Capital and Brightmind Partners doubling down, alongside top VCs including Theory Ventures, Lockstep, Two Sigma Ventures, and prominent cybersecurity industry leaders, including the founders of Abnormal AI and Demisto, the former CEO and CTO of Splunk, and senior executives from CrowdStrike, Palo Alto Networks, Microsoft, and Okta. I spent a decade building enterprise security platforms at Demisto, Palo Alto Networks, and AWS GuardDuty. Dan led AI/ML at Abnormal AI and Twitter. Across hundreds of security teams, we kept seeing the same thing: more data than ever, and almost no way to use it to stop attacks. Detections tuned for someone else's environment. Investigations done by hand. Attackers moving faster than any human workflow can match. We built Artemis to change that. Artemis understands your environment before it starts looking for threats - modeling relationships between users, machines, cloud workloads, applications, and business context across every log source. Detections adapt to your environment. Every signal triggers an autonomous investigation that already knows your baseline. You don't get an alert and a link to raw logs. You get a story: what happened, the evidence, what was contained by Artemis, and what needs your judgment. The result: our customers have reduced their mean time to detect and respond to security events by 94%. We're in production with some of the world's largest companies across financial services, technology, and insurance, analyzing over 15,000TB daily. We are grateful for those who believed from the first moment till now and excited for what’s to come. If your security team is spending more time assembling context than making decisions, we'd love to show you what Artemis can do. Want to learn more? Visit https://lnkd.in/ejKq3zik
-
Pavel Zhiriakov ha recomendado estoPavel Zhiriakov ha recomendado estoWe just brought Wix and Base44 directly into Microsoft Copilot, partnering with Microsoft. Soon you’ll be able to create a full Wix Harmony website and apps with Base44 directly from a conversation. Just describe your idea and start building. We’re continuing to bring users the ability to build a website from wherever you are. https://lnkd.in/dq9HX532Enable agents to bring apps into the flow of work—while keeping IT in control | Microsoft Community HubEnable agents to bring apps into the flow of work—while keeping IT in control | Microsoft Community Hub
-
Pavel Zhiriakov ha recomendado estoPavel Zhiriakov ha recomendado estoAfter 6.5 unforgettable years, my journey at Wix has come to a close, and I am thrilled to finally share what’s next! When I first joined Wix, I stepped in to manage 10 data engineers. Together, we took that foundation, we grew significantly and we built it into a powerhouse Data Engineering Guild. Watching this community evolve into a professional benchmark, both inside and outside the company, has been the greatest achievement of my career so far. My time at Wix gave me the freedom to explore and fail, but the greatest gift it gave me was clarity. It helped me realize that my true passion is building. I thrive when I can take existing pieces with huge potential and build the overarching vision and structure that brings them all together. Because of that, I knew my next move had to reflect that exact passion. I am so excited to announce that I am joining Nagomi Security as their first-ever Head of Data! On day one, I’ll be stepping in to manage two incredible existing teams. My focus will be on building the unified data strategy, culture, and architecture to help drive Nagomi's next phase of growth. I can't wait to dive in and get started. Of course, closing this chapter means saying goodbye to the people who made it so special. A company’s true superpower is its talent, and Wix is full of brilliant minds: - A massive public thank you to my manager, Timor Wienrib, and my HR partner, Yael Kaminsky. You two are the ones who truly made me who I am today as a leader. Thank you for your endless guidance, your trust, and for being such incredible partners in this journey. I wouldn't be where I am without you. - A special shoutout to Josef Goldstein, you were the best partner I could have asked for. I learned so much from you, you constantly inspired me, and beyond our great collaborations, I am so deeply grateful for your invaluable advice and true friendship. - To the amazing team leads I’ve had the privilege of managing: you have been my home and my family. I’m incredibly proud of what we achieved, and I’m taking so many of you with me in my heart. Passing the baton is much easier knowing the Guild is being taken over by an absolute dream team. Hila Sofer Elyashiv, Moti P., and Eden Bar-Tov, I have complete faith in you, and I’ll be cheering you on from the sidelines! Beyond these names, there are so many other incredible people across Wix who I’ve worked with, learned from, and built alongside. Thank you for the ride of a lifetime. I am taking a piece of this amazing community with me as I step into this next chapter. To my entire network: my inbox is always open, so feel free to reach out. Here’s to the next build! 🚀 #DataEngineering #HeadOfData #Leadership #NextChapter #NagomiSecurity #Wix #Building
Experiencia y educación
Licencias y certificaciones
Cursos
-
Business Intelligence
COURSERA
-
Oracle SQL 12c
Специалист, Oracle SQL Pr
Proyectos
-
Geospatial Data Tool for Real Estate Insights
-
**Project Overview:**
Recently, I collaborated with a partner to develop a tool for accessing property data in Tel Aviv, targeting B2B clients like real estate agencies.
Project Description:
Our tool retrieves detailed property information using a Telegram bot and a REST API developed with FastAPI. Users can get data on building details, such as the year of construction and number of floors. We integrated additional data on trees, parks, kindergartens, shelters, beaches, and air…**Project Overview:**
Recently, I collaborated with a partner to develop a tool for accessing property data in Tel Aviv, targeting B2B clients like real estate agencies.
Project Description:
Our tool retrieves detailed property information using a Telegram bot and a REST API developed with FastAPI. Users can get data on building details, such as the year of construction and number of floors. We integrated additional data on trees, parks, kindergartens, shelters, beaches, and air quality into a PostGIS-powered Data Warehouse (DWH).
We used Tel Aviv’s public API (ESRI GIS) and other APIs, employing machine learning techniques like classification and kernel density estimation. We scraped data from sites like yad2 using BeautifulSoup, Chromium, Puppeteer, Selenium, and Playwright. Our goal was to create clusters indicating whether a property's price is below average, normal, or above average.
Technologies and Tools:
- *Programming Languages & Libraries:* Python, Geopandas, Polars, BeautifulSoup
- *Databases:* PostGIS
- *Data Analysis & Visualization:* QGIS, Jupyter Notebooks
- *Asynchronous Programming:* asyncio
- *ORM:* SQLAlchemy
- *APIs & Data Retrieval:* Requests, Public APIs (ESRI GIS), Scraping APIs
- *Frontend:* Telegram bot, REST API (FastAPI)
- *Containerization:* Docker
*Impact and Conclusion:*
Although we didn't achieve a perfect product-market fit, this project was a significant learning experience. It showcased my ability to design data pipelines, implement machine learning techniques, and develop user-centric products. This project highlights my skills in data engineering, geospatial analysis, and product development, demonstrating my passion for creating impactful solutions.
Calificaciones de pruebas
-
ÖSD Sprachdiplom | B2
Puntuación: Bestanden
Deutsche sprache
Idiomas
-
English
Competencia básica profesional
-
Deutsch
Competencia básica profesional
-
Russian
Competencia bilingüe o nativa
-
Hebrew
Competencia básica