Hey Folks!

This newsletter will discuss some cool AI research papers and AI tools. Happy learning!

👉 What is Trending in AI/ML Research? 

➡️ This AI Research by Microsoft and Tsinghua University Introduces EvoPrompt: A Novel AI Framework for Automatic Discrete Prompt Optimization Connecting LLMs and Evolutionary Algorithms

How can we automate the process of generating effective prompts for Large Language Models (LLMs) without intensive human effort? This paper introduces EvoPrompt, a groundbreaking framework that fuses the strengths of evolutionary algorithms (EAs) and LLMs to optimize discrete, coherent, human-readable prompts. Foregoing gradients and parameters, EvoPrompt initiates with a prompt population, then utilizes LLMs to iteratively produce new prompts influenced by evolutionary operators, refining based on a development set. Testing on both closed- and open-source LLMs, including GPT-3.5 and Alpaca, across 9 datasets, EvoPrompt outperformed human-crafted and other automatic prompt generation methods, spotlighting the potential of merging LLMs with traditional algorithms.

➡️  ReLU vs. Softmax in Vision Transformers: Does Sequence Length Matter? Insights from a Google DeepMind Research Paper

How can one maintain accuracy when substituting the attention softmax with a point-wise activation in vision transformers? This study reveals that the previously observed accuracy degradation can be alleviated by dividing by sequence length. By training vision transformers of varying sizes on ImageNet-21k, the researchers demonstrate that ReLU-attention, when adjusted this way, can match or even rival the performance of softmax-attention in scalability relative to compute resources.

➡️  OpenAI Unveils DALL·E 3: A Revolutionary Leap in Text-to-Image Generation

In a significant technological leap, OpenAI has announced the launch of DALL·E 3, the latest iteration in their groundbreaking text-to-image generation technology. With an unprecedented capacity to understand nuanced and detailed descriptions, DALL·E 3 promises to revolutionize the creative landscape by allowing users to translate their textual ideas into astonishingly accurate images effortlessly. DALL·E 3 is currently in research preview, offering a tantalizing glimpse into its capabilities. However, the broader availability of this cutting-edge technology is set for early October, when it will be accessible to ChatGPT Plus and Enterprise customers through the API and Labs later in the fall.

➡️ Do Machine Learning Models Produce Reliable Results with Limited Training Data? This New AI Research from Cambridge and Cornell University Finds it..

How can we confidently recover unknown physical systems using limited data in the realm of Partial Differential Equations (PDE) learning? Building upon the interplay between physics and machine learning, this paper offers theoretical assurances on the minimal input-output training pairs necessary for PDE learning. By harnessing the power of randomized numerical linear algebra combined with PDE theory, the researchers propose a data-efficient algorithm. This novel approach is designed to retrieve solution operators of three-dimensional uniformly elliptic PDEs from sparse input-output datasets. Remarkably, it promises an exponential decline in error rate correlated to the training dataset size, ensuring a high success likelihood.

➡️ Researchers from the University of Oregon and Adobe Introduce CulturaX: A Multilingual Dataset with 6.3T Tokens in 167 Languages Tailored for Large Language Model (LLM) Development

How can the research community address the issue of lack of transparency in training datasets for large language models (LLMs), especially when biases and hallucinations are of concern and there's an evident scarcity of high-quality multilingual datasets? This paper introduces "CulturaX," a massive multilingual dataset tailored for LLM development, encompassing 6.3 trillion tokens across 167 languages. CulturaX undergoes a stringent cleaning and deduplication process, employing stages like language identification, URL-based filtering, metric-based cleaning, document refinement, and data deduplication. By making CulturaX available on HuggingFace, the authors aim to boost research and progress in multilingual LLM development.

👉 What is Trending in AI Tools?