06:03PM EDT - Cerebras did the wafer scale - a single chip the size of a wafer

06:03PM EDT - Here's WSE1

06:04PM EDT - Uses standard tools like TensorFlow and pyTorch with Cerebras compiler

06:04PM EDT - CS-1 fits in standard 15U rack

06:04PM EDT - 400k cores

06:04PM EDT - (Costs a few $mil each)

06:05PM EDT - No DRAM, full on-chip SRAM

06:05PM EDT - 3D mesh network

06:06PM EDT - Allows all 400k cores to work on the same problem

06:06PM EDT - Linear perf scaling

06:07PM EDT - Cerebras graph compiler

06:07PM EDT - extract compute graph, create a graph in the WSE format, route kernals on the fabric, then create executable

06:09PM EDT - Graph matching for matrix multiply loops

06:09PM EDT - Supports hand optimized kernels

06:10PM EDT - Model-parallel and data-parallel optimization with the compiler

06:11PM EDT - Trade-off as resources vs compute for each kernel

06:12PM EDT - All kernels can be resized as needed

06:12PM EDT - All functionally identical

06:13PM EDT - Global optimization function to maximize throughput and utilization

06:14PM EDT - 3 key benefits

06:15PM EDT - Flexible parallelism

06:16PM EDT - Enough Fabric performance to connect everything at scale

06:16PM EDT - Otherwise slow across GPUs or a cluster

06:16PM EDT - Small batch size has super high utilization

06:16PM EDT - no weight sync overhead

06:19PM EDT - Core is designed for sparsity

06:20PM EDT - Intrinsic sparsity harvesting

06:20PM EDT - filters out all zeros

06:23PM EDT - ML user has full control over full range of sparse techniques

06:23PM EDT - WSE is MIMD, each core can be independent

06:24PM EDT - True variable sequence length support

06:24PM EDT - No padding required

06:24PM EDT - Higher utilization for irregular models

06:25PM EDT - Dynamic depth networks

06:26PM EDT - only need to process exact lengths of sequences

06:26PM EDT - World's most powerful AI computer

06:27PM EDT - Complete flexibility due to the size of the wafer scale engine

06:28PM EDT - Working in the lab today

06:29PM EDT - More info later this year

06:29PM EDT - Q&A team

06:29PM EDT - time*

06:31PM EDT - Q: What's the main benefit of WSE? A: Bypassing the issues with workloads that need multiple GPU/TPU/DPU. Opens up novel techniques that wouldn't run on traditional hardware at any sense of speed

06:32PM EDT - Q: How to feed the beast? A: Traditional server might be the GPU/TPU, so the next one is IO. We are a system company, our product is the full system, as we control all aspects of the system. We have a 1.2 Tb/s ethernet interconnect to feed the engine to keep up with the compute

06:34PM EDT - Q: How long does it take a compile a model over 400k units? A: It's an algorithmically complex search space problem. Annealing and heuristics bring that down - we are borrowing many ideas from the EDA industry. Our problem is simpler than billions of LEs on FPGAs, so we're in the minutes.

06:34PM EDT - That's a wrap. Next talk is 4096 RISC-V chip



View All Comments

  • Technical services - Sunday, January 3, 2021 - link

    The villa is a huge, separate structure with vast land surrounding generally having three or further floors that distinguish it from a bungalow. Villas are usually constructed in Dubai for various purposes. Villa needs maintenance and fit-out services after a regular interval of time. For this purpose, Abdulla Thani group of the technical services are providing <a href="https://technicalservice.ae/villa-fit-out-and-pain... villa fit-out and painting services.</a> Dubai. Abdulla Thani's group of technical services continually ranking in top positions for its outstanding services and teamwork. Reply
  • Daily Medicos - Thursday, January 14, 2021 - link

    Hello! Hope you are doing well. Daily Medicos is a content hub that provides articles on Health & Wellness, prevention from diseases, Guidance for medical conditions, Healthy Nutritional tips, inspiration on mental health problems, and a Healthy living lifestyle! Here you find all the <a href="https://dailymedicos.com/simple-healthy-tips-for-e... remedies for bruised toenail</a>, how to remove the stretch marks, and everything related to health and medical issues. Reply
  • Digital00 - Friday, January 15, 2021 - link

    DRD Digital Marketing Agency is the Best Digital Marketing Agency in London. They provide the best services for business and corporates. DRD offers services like Social Media, Web Design, Web Traffic, SEO, App development, Graphic Design, and many more other services. If you want to avail of any of these services, go and check out their website.
  • Fabcity - Tuesday, January 26, 2021 - link

    Fabcity, The largest store for online shopping in Nigeria to provide thousands of products at one stop like Mobiles, Cameras, Fashion, Computers And More.
    Visit us more detail : www.fabcity.ng
  • Fabcity - Tuesday, January 26, 2021 - link

    Fabcity, The largest store for online shopping in Nigeria to provide thousands of products at one stop like Mobiles, Cameras, Fashion, Computers And More.
    Visit us more detail : https://www.fabcity.ng/
  • plenty999 - Friday, January 29, 2021 - link

    Plenty Thingz is an E-Commerce store. It provides you the best thing for your child to encourage your child to play inside like pool dome and many more. It also provides you this cool and fun scooter for your child. It's an excellent toy to encourage your kids to play outside. It is great for improving strength, coordination, and balance.
  • vantage000 - Thursday, February 4, 2021 - link

    Vantage Security Systems will provide you with a complete solution to secure and monitor your business. After evaluating your needs, we will come up with a cost effective solution that will give you the piece of mind you are looking for.
  • Kraktech Digital - Tuesday, February 9, 2021 - link

    Do you want a high quality wordpress website design? The design of your Website is your first expression for your customer. Kraktech digitals provides you wordpress website designs for your business and offer amazing features for our customers to create a masterpiece for them. We ensure 100% customer satisfaction from our design. You will surely enjoy working with Kraktech
  • miami009 - Tuesday, February 9, 2021 - link

    Miami Laundry Company Wash offer convenient and exclusive services in Miami with a budget-family and inexpensive cost
  • Utubster - Monday, February 15, 2021 - link

    Download playlists, videos and MP3s from major websites like never before. We designed and tested the best features a YouTube Downloader should have. Together, we shaped the new standard

Log in

Don't have an account? Sign up now