Post
59
No Cuda - No 4GB of Python Dependencies - Pure Pascal + AVX + OpenCL in a Single Binary
Dr. Joao Paulo Schwarz Schuler PRO
schuler
AI & ML interests
artificial intelligence
Recent Activity
posted an update 3 months ago
**Everyone says Pascal died in the 90s.** Meanwhile, this Free Pascal library runs GPT-style transformers, Mixture-of-Experts, RWKV, xLSTM, spiking neurons, and normalizing flows, natively, with no Python nor CUDA lock-in.
These models have been tested and they run in pure Free Pascal:
* HuggingFaceTB/SmolLM2-1.7B-Instruct
* Qwen/Qwen2.5-{0.5B,1.5B,3B,7B,14B,32B}-Instruct
* Qwen/Qwen3-0.6B
* Qwen/Qwen3-30B-A3B-Thinking-2507
* TinyLlama/TinyLlama-1.1B-Chat-v1.0
* mistralai/Mistral-7B-Instruct-v0.3
* microsoft/Phi-3-mini-4k-instruct
* allenai/OLMoE-1B-7B-0125-Instruct
* state-spaces/mamba-130m-hf
* openai-community/gpt2
https://github.com/joaopauloschuler/neural-api
reacted to ajibawa-2023's post with ๐ 8 months ago
PHP-Code-Large
Dataset: https://huggingface.co/datasets/ajibawa-2023/PHP-Code-Large
PHP-Code-Large is a large-scale corpus of PHP source code comprising more than 12 million lines of PHP code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and static program analysis for the PHP ecosystem.
By providing a high-volume, language-specific corpus, PHP-Code-Large enables systematic experimentation in PHP-focused model training, domain adaptation, and downstream code understanding tasks.
PHP-Code-Large addresses the need for a dedicated PHP-only dataset at substantial scale, enabling focused research across backend systems, CMS platforms, APIs, and full-stack PHP environments.Organizations
None yet
posted an update 5 days ago
posted an update 3 months ago
Post
127
**Everyone says Pascal died in the 90s.** Meanwhile, this Free Pascal library runs GPT-style transformers, Mixture-of-Experts, RWKV, xLSTM, spiking neurons, and normalizing flows, natively, with no Python nor CUDA lock-in.
These models have been tested and they run in pure Free Pascal:
* HuggingFaceTB/SmolLM2-1.7B-Instruct
* Qwen/Qwen2.5-{0.5B,1.5B,3B,7B,14B,32B}-Instruct
* Qwen/Qwen3-0.6B
* Qwen/Qwen3-30B-A3B-Thinking-2507
* TinyLlama/TinyLlama-1.1B-Chat-v1.0
* mistralai/Mistral-7B-Instruct-v0.3
* microsoft/Phi-3-mini-4k-instruct
* allenai/OLMoE-1B-7B-0125-Instruct
* state-spaces/mamba-130m-hf
* openai-community/gpt2
https://github.com/joaopauloschuler/neural-api
These models have been tested and they run in pure Free Pascal:
* HuggingFaceTB/SmolLM2-1.7B-Instruct
* Qwen/Qwen2.5-{0.5B,1.5B,3B,7B,14B,32B}-Instruct
* Qwen/Qwen3-0.6B
* Qwen/Qwen3-30B-A3B-Thinking-2507
* TinyLlama/TinyLlama-1.1B-Chat-v1.0
* mistralai/Mistral-7B-Instruct-v0.3
* microsoft/Phi-3-mini-4k-instruct
* allenai/OLMoE-1B-7B-0125-Instruct
* state-spaces/mamba-130m-hf
* openai-community/gpt2
https://github.com/joaopauloschuler/neural-api
reacted to ajibawa-2023's post with ๐ 8 months ago
Post
2,601
PHP-Code-Large
Dataset: ajibawa-2023/PHP-Code-Large
PHP-Code-Large is a large-scale corpus of PHP source code comprising more than 12 million lines of PHP code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and static program analysis for the PHP ecosystem.
By providing a high-volume, language-specific corpus, PHP-Code-Large enables systematic experimentation in PHP-focused model training, domain adaptation, and downstream code understanding tasks.
PHP-Code-Large addresses the need for a dedicated PHP-only dataset at substantial scale, enabling focused research across backend systems, CMS platforms, APIs, and full-stack PHP environments.
Dataset: ajibawa-2023/PHP-Code-Large
PHP-Code-Large is a large-scale corpus of PHP source code comprising more than 12 million lines of PHP code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and static program analysis for the PHP ecosystem.
By providing a high-volume, language-specific corpus, PHP-Code-Large enables systematic experimentation in PHP-focused model training, domain adaptation, and downstream code understanding tasks.
PHP-Code-Large addresses the need for a dedicated PHP-only dataset at substantial scale, enabling focused research across backend systems, CMS platforms, APIs, and full-stack PHP environments.
posted an update 8 months ago
Post
2,232
โก Speaking is faster than typing - BPSA - Powerful Open Source Agentic Coding Tool with Voice Support - a smolagents fork - Interactive REPL CLI Tool - New Voice Support!
โฟ Accessibility โ It's a huge benefit for people with mobility issues, dyslexia, or other conditions that make typing difficult.
๐๏ธ Hands-free operation โ You can dictate while doing other things, like looking at documents or moving around.
https://github.com/joaopauloschuler/beyond-python-smolagents/
โฟ Accessibility โ It's a huge benefit for people with mobility issues, dyslexia, or other conditions that make typing difficult.
๐๏ธ Hands-free operation โ You can dictate while doing other things, like looking at documents or moving around.
https://github.com/joaopauloschuler/beyond-python-smolagents/
posted an update 8 months ago
Post
221
โก Install AI-powered coding assistance in under 1 minute - watch the quick demo to see how easy setup can be.
https://github.com/joaopauloschuler/beyond-python-smolagents
https://github.com/joaopauloschuler/beyond-python-smolagents
posted an update 9 months ago
Post
194
BPSA - a smolagents fork - Powerful Interactive REPL CLI Tool for Agentic Coding
https://github.com/joaopauloschuler/beyond-python-smolagents?tab=readme-ov-file#cli-bpsa
Includes context compression and plenty of file manipulation tools to make efficient context usage.
https://github.com/joaopauloschuler/beyond-python-smolagents?tab=readme-ov-file#cli-bpsa
Includes context compression and plenty of file manipulation tools to make efficient context usage.
replied to their post 9 months ago
I finished coding the first version of bp_cli.py.
replied to their post 9 months ago
I started coding it. It has context compression and plenty other tricks:
https://github.com/joaopauloschuler/beyond-python-smolagents/blob/development6/src/smolagents/bp_cli.py
replied to their post 9 months ago
This is an excellent idea...
posted an update 9 months ago
Post
226
๐ Excited to share the most starred fork of @huggingface 's smolagents with game-changing additions:
๐๏ธ Context compression - LLM-based summarization for long-running tasks
โก Native Python execution via exec - no restrictions
๐ Infinite runtime - agents that keep coding/working from a single prompt
๐ Multi-language support - Pascal, PHP, C++, Java & more
๐ ๏ธ Developer agent tools - compile, test, debug across languages
๐ฅ Multi-agent collaboration - tackle complex problems together
๐ Colab integration - use google.colab.ai models
Check it out on GitHub โญ
https://github.com/joaopauloschuler/beyond-python-smolagents
๐๏ธ Context compression - LLM-based summarization for long-running tasks
โก Native Python execution via exec - no restrictions
๐ Infinite runtime - agents that keep coding/working from a single prompt
๐ Multi-language support - Pascal, PHP, C++, Java & more
๐ ๏ธ Developer agent tools - compile, test, debug across languages
๐ฅ Multi-agent collaboration - tackle complex problems together
๐ Colab integration - use google.colab.ai models
Check it out on GitHub โญ
https://github.com/joaopauloschuler/beyond-python-smolagents
- 4 replies
posted an update over 1 year ago
Post
447
๐ข I will create your README.md file 4 you 4 free
I am currently testing an open source prototype that writes/updates README.md files for projects with up to 10k source code lines.
If you have an open source project needing a new README file on github, just reply to this post with your project URL. Your project will be used in my testing. If all is good in the testing, I'll send you a pull request via github with the updated README.md.
No submitted source code will be used for model training.
All computer languages are accepted including pascal, java, cobol, fortran, lisp, โฆ
I will create READMEs only for the first few requests. The testing is available only to open source projects.
I am currently testing an open source prototype that writes/updates README.md files for projects with up to 10k source code lines.
If you have an open source project needing a new README file on github, just reply to this post with your project URL. Your project will be used in my testing. If all is good in the testing, I'll send you a pull request via github with the updated README.md.
No submitted source code will be used for model training.
All computer languages are accepted including pascal, java, cobol, fortran, lisp, โฆ
I will create READMEs only for the first few requests. The testing is available only to open source projects.
reacted to AdinaY's post with ๐ over 1 year ago
Post
2,578
Two AI startups, DeepSeek & Moonshot AI , keep moving in perfect sync ๐
โจ Last December: DeepSeek & Moonshot AI released their reasoning models on the SAME DAY.
DeepSeek: deepseek-ai/DeepSeek-R1
MoonShot: https://github.com/MoonshotAI/Kimi-k1.5
โจ Last week: Both teams published papers on modifying attention mechanisms on the SAME DAY AGAIN.
DeepSeek: Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (2502.11089)
Moonshot: MoBA: Mixture of Block Attention for Long-Context LLMs (2502.13189)
โจ TODAY:
DeepSeek unveiled Flash MLA: a efficient MLA decoding kernel for NVIDIA Hopper GPUs, optimized for variable-length sequences.
https://github.com/deepseek-ai/FlashMLA
Moonshot AI introduces Moonlight: a 3B/16B MoE trained on 5.7T tokens using Muon, pushing the Pareto frontier with fewer FLOPs.
moonshotai/Moonlight-16B-A3B
What's next? ๐
โจ Last December: DeepSeek & Moonshot AI released their reasoning models on the SAME DAY.
DeepSeek: deepseek-ai/DeepSeek-R1
MoonShot: https://github.com/MoonshotAI/Kimi-k1.5
โจ Last week: Both teams published papers on modifying attention mechanisms on the SAME DAY AGAIN.
DeepSeek: Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (2502.11089)
Moonshot: MoBA: Mixture of Block Attention for Long-Context LLMs (2502.13189)
โจ TODAY:
DeepSeek unveiled Flash MLA: a efficient MLA decoding kernel for NVIDIA Hopper GPUs, optimized for variable-length sequences.
https://github.com/deepseek-ai/FlashMLA
Moonshot AI introduces Moonlight: a 3B/16B MoE trained on 5.7T tokens using Muon, pushing the Pareto frontier with fewer FLOPs.
moonshotai/Moonlight-16B-A3B
What's next? ๐
reacted to onekq's post with ๐ over 1 year ago
Post
2,456
Huge disappointment to Claude Sonnet 3.7 ๐ Big performance regression. Worse than the June version in 2024. ๐
onekq-ai/WebApp1K-models-leaderboard
I'm sure though this version improves on something, only not the thing my leaderboard measures. This proves the point that no model can be the best on everything.
onekq-ai/WebApp1K-models-leaderboard
I'm sure though this version improves on something, only not the thing my leaderboard measures. This proves the point that no model can be the best on everything.
- 2 replies
posted an update over 1 year ago
Post
2,073
๐ข Old Research Alert: Making Computer Vision Models Smaller & Smarter!
Years ago, I coded an optimization in the first layers of a convolutional neural network (computer vision) and ended never posting here. The optimization decreases the number of parameters while increasing accuracy. The optimization relies in separating (branching) chromatic and achromatic information through the layers of a neural network.
YouTube videos:
https://www.youtube.com/watch?v=u4vZZmBMFLw
https://www.youtube.com/watch?v=-BD293yqdKI
Source codes:
https://github.com/joaopauloschuler/two-branch-plant-disease
https://github.com/joaopauloschuler/two-path-noise-lab-plant-disease
Research papers:
https://www.researchgate.net/publication/361511874_Color-Aware_Two-Branch_DCNN_for_Efficient_Plant_Disease_Classification
https://www.researchgate.net/publication/355215213_Reliable_Deep_Learning_Plant_Leaf_Disease_Classification_Based_on_Light-Chroma_Separated_Branches
May the force be with you.
Years ago, I coded an optimization in the first layers of a convolutional neural network (computer vision) and ended never posting here. The optimization decreases the number of parameters while increasing accuracy. The optimization relies in separating (branching) chromatic and achromatic information through the layers of a neural network.
YouTube videos:
https://www.youtube.com/watch?v=u4vZZmBMFLw
https://www.youtube.com/watch?v=-BD293yqdKI
Source codes:
https://github.com/joaopauloschuler/two-branch-plant-disease
https://github.com/joaopauloschuler/two-path-noise-lab-plant-disease
Research papers:
https://www.researchgate.net/publication/361511874_Color-Aware_Two-Branch_DCNN_for_Efficient_Plant_Disease_Classification
https://www.researchgate.net/publication/355215213_Reliable_Deep_Learning_Plant_Leaf_Disease_Classification_Based_on_Light-Chroma_Separated_Branches
May the force be with you.
posted an update over 1 year ago
Post
3,438
๐ฎ GPT-3 implemented in pure Free Pascal!
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
Clean Pascal Implementation
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
โโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Input Layer โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Token & Positional โ
โ Embedding โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 12x Transformer โ
โ Blocks โ
โ - 12 heads โ
โ - 768 hidden dims โ
โ - 3072 intermediate โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Output Layer โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโClean Pascal Implementation
for CntLayer := 1 to {Layers=}12 do
begin
Result.AddTransformerBlockCAI(
{Heads=}12,
{intermediate dimensions=}4*768,
{NoForward=}true,
{HasNorm=}true,
false
);
end;replied to their post over 1 year ago
In the case that you run into any roadblock at modifying an existing model with this optimization so you can train the optimized model from scratch, please feel free to ask for help.
posted an update over 1 year ago
Post
7,279
๐ข New Research Alert: Making Language Models Smaller & Smarter!
Thrilled to share the latest technical report demonstrating how to reduce language model parameters by 77% while maintaining performance.
The secret? Grouped pointwise convolutions. Yes. We brought a method from computer vision to the transformers arena.
๐ Key Findings:
โข 77% parameter reduction.
โข Maintained model capabilities.
โข Improved generalization.
Paper: https://www.researchgate.net/publication/388835829_SAVING_77_OF_THE_PARAMETERS_IN_LARGE_LANGUAGE_MODELS_TECHNICAL_REPORT
Code: https://github.com/joaopauloschuler/less-parameters-llm
Thrilled to share the latest technical report demonstrating how to reduce language model parameters by 77% while maintaining performance.
The secret? Grouped pointwise convolutions. Yes. We brought a method from computer vision to the transformers arena.
๐ Key Findings:
โข 77% parameter reduction.
โข Maintained model capabilities.
โข Improved generalization.
Paper: https://www.researchgate.net/publication/388835829_SAVING_77_OF_THE_PARAMETERS_IN_LARGE_LANGUAGE_MODELS_TECHNICAL_REPORT
Code: https://github.com/joaopauloschuler/less-parameters-llm
- 2 replies