Alexandr Ryabchenko

If you’ve been following SVD-based compression methods (SVD-LLM, Basis Sharing), the relationships between them have been murky. Since last two months I have been rigorously trying to prove that they are all just corners of a single, 2D design space—specifically, an optimization problem on a Grassmannian fiber bundle. I mathematically characterized this entire space and found the absolute optimal configuration. In terms of pure linear algebra, it’s flawless: I achieved a 46.4% reduction in Frobenius reconstruction error over SVD-LLM on Pythia. But here is where the theory shattered against reality. When I tested this mathematically "perfect" compression on downstream tasks, performance collapsed. At identical compression ratios, perplexity blew up (from 50.17 to 159.14) and accuracy dropped by 13%. What went wrong? Mechanistically, the bundle energy tightly couples adjacent layers through their weights. But in a deployed transformer, the residual stream decouples them. By perfectly optimizing across layers (the Frobenius gain), I unintentionally sacrificed per-layer optimality. The Bottom Line: Cross-layer Frobenius optimization is a trap for LLM compression. Downstream quality tracks per-layer activation reconstruction, not weight-space metrics. Future cross-layer methods must pivot their core objectives if they want mathematical gains to actually translate to deployment. Dive into the theorems, the phase diagrams, and my diagnostic sweeps below: https://lnkd.in/gePevXUY #MachineLearning #LLMs #ArtificialIntelligence #ModelCompression #AI

10

my slides on Ambiguous vs. Unambiguous Grammar and Lexeme & Tokenization. Working on these concepts gave me a much deeper understanding of Compiler Construction. A big thank you to Mam Zobia Zafar for her guidance, and to my teammate for the collaboration.Basim Qamar Always learning, always growing. #Education #CompilerConstruction #Teamwork #studentlife

20

I'm pleased to welcome Yannik Pitcan, PhD to AdZeta's founding team as a Senior ML Engineer & Signals architect. Yannik is a Ph.D. in Statistics from University of California, Berkeley & A.B. in Mathematics from Harvard University. He is also an ex-data scientist at Google where he built attribution pipelines processing billions of events. As we build a category-defining product in the ad signals space and support that with the infrastructure that allows DTC brands to bid on future value, best-in-class engineering talent is what we need to accomplish our vision. At AdZeta, Yannik is taking ownership of our ML architecture and signal calibration engine. He will be bridging the gap between raw ML outputs and ad network ingestion, using advanced Bayesian modeling and causal inference to ensure the pLTV signals we fire into the ad networks are perfectly calibrated for their native auction dynamics. Welcome aboard, Yannik! Let’s build. 

48

4 комментария

I’m working as a Software Engineer at Facebook (Meta) with over 20 years of experience. If I were beginning my career again in 2026 and wanted to become an ML Engineer, these are the skills I would master first… [1] Python as the main language - Write Python every day, focus on clean functions, classes and modules. - Automate boring tasks like data cleaning, file handling and API calls. - Pick one backend stack, for example FastAPI or Django, and build simple APIs for your models. [2] Core machine learning fundamentals - Learn supervised and unsupervised learning, loss functions and metrics like accuracy, F1 and AUC. - Implement key algorithms from scratch in Python, such as linear regression, logistic regression, decision trees and k-means. - Take real job descriptions and map each requirement to a concept or method you know. [3] Deep learning and frameworks - Pick PyTorch and learn tensors, modules, optimizers and training loops. - Build small projects in vision and text, for example image classification and sentiment analysis. - Recreate a few public projects end to end, from raw data to a trained model with clear metrics. [4] Software engineering and production thinking - Use git every day, write tests, add logging and handle errors in a clear way. - Design simple services that load a model once and serve fast predictions through an API. - Turn your experiments into repeatable pipelines with configs, scripts and a fixed folder structure. [5] ML lifecycle and MLOps - Track experiments and models with tools like MLflow, or with a clear manual system at the start. - Learn Docker and package a small ML service into a container you can run anywhere. - Schedule training and batch inference jobs with simple tools, then move to managed cloud services as you grow. [6] AI and LLM skills - Learn NLP basics, tokenization, embeddings and how to evaluate text models. - Use LLMs to build small features such as summarization, classification or simple chat flows. - Practice prompt design and learn at least one method like fine tuning or retrieval to adapt models to real tasks. [7] Communication and soft skills - Explain every project with three points: problem, approach, and impact. - Write short docs for your work and treat them as part of your portfolio. - Practice speaking your thinking during mock interviews so your reasoning stays clear while you code. [8] Cloud foundations - Pick AWS and learn core services like S3, EC2, and one database option. - Deploy at least one ML service to the cloud, even if it is simple. - Learn basic cost and reliability trade-offs so your designs stay lean and practical.

440

19 комментариев

If you use affordable open source AI models for coding, you may notice that they often struggle with code quality and debugging. I have been working on something that can help. Open CCMD (Claude Code Markdown) is a collection of expert level coding assistant guides (CLAUDE.md files) designed to help open source AI models write cleaner code, debug more effectively, and follow best practices. What it provides ✅ Systematic approaches to code generation ✅ Better debugging with hypothesis driven methods ✅ Enforced best practices and patterns ✅ Guidance for understanding codebases before making changes ✅ Support for proper testing and verification Supported in 9 languages C, C++, C#, Go, Java, JavaScript, Python, Rust, TypeScript These guides improve the performance of open source models, though they do not yet reach the level of premium models like Claude Opus, Sonnet, or GPT 5. This project is one step toward closing that gap. The project is open source, MIT licensed, and ready to use. Add the guide for your language as CLAUDE.md in your project and let your AI assistant reference it. 🔗 Project link: https://lnkd.in/gyCEpMB4 Contributions are welcome. If you want to add a language/framework or improve the guides, feel free to join. #OpenSource #AICoding #MachineLearning #SoftwareDevelopment #CodingAssistance #AI #DeveloperTools #OpenSourceAI #Coding #Programming

27

1 комментарий