06:03PM EDT - Cerebras did the wafer scale - a single chip the size of a wafer

06:03PM EDT - Here's WSE1

06:04PM EDT - Uses standard tools like TensorFlow and pyTorch with Cerebras compiler

06:04PM EDT - CS-1 fits in standard 15U rack

06:04PM EDT - 400k cores

06:04PM EDT - (Costs a few $mil each)

06:05PM EDT - No DRAM, full on-chip SRAM

06:05PM EDT - 3D mesh network

06:06PM EDT - Allows all 400k cores to work on the same problem

06:06PM EDT - Linear perf scaling

06:07PM EDT - Cerebras graph compiler

06:07PM EDT - extract compute graph, create a graph in the WSE format, route kernals on the fabric, then create executable

06:09PM EDT - Graph matching for matrix multiply loops

06:09PM EDT - Supports hand optimized kernels

06:10PM EDT - Model-parallel and data-parallel optimization with the compiler

06:11PM EDT - Trade-off as resources vs compute for each kernel

06:12PM EDT - All kernels can be resized as needed

06:12PM EDT - All functionally identical

06:13PM EDT - Global optimization function to maximize throughput and utilization

06:14PM EDT - 3 key benefits

06:15PM EDT - Flexible parallelism

06:16PM EDT - Enough Fabric performance to connect everything at scale

06:16PM EDT - Otherwise slow across GPUs or a cluster

06:16PM EDT - Small batch size has super high utilization

06:16PM EDT - no weight sync overhead

06:19PM EDT - Core is designed for sparsity

06:20PM EDT - Intrinsic sparsity harvesting

06:20PM EDT - filters out all zeros

06:23PM EDT - ML user has full control over full range of sparse techniques

06:23PM EDT - WSE is MIMD, each core can be independent

06:24PM EDT - True variable sequence length support

06:24PM EDT - No padding required

06:24PM EDT - Higher utilization for irregular models

06:25PM EDT - Dynamic depth networks

06:26PM EDT - only need to process exact lengths of sequences

06:26PM EDT - World's most powerful AI computer

06:27PM EDT - Complete flexibility due to the size of the wafer scale engine

06:28PM EDT - Working in the lab today

06:29PM EDT - More info later this year

06:29PM EDT - Q&A team

06:29PM EDT - time*

06:31PM EDT - Q: What's the main benefit of WSE? A: Bypassing the issues with workloads that need multiple GPU/TPU/DPU. Opens up novel techniques that wouldn't run on traditional hardware at any sense of speed

06:32PM EDT - Q: How to feed the beast? A: Traditional server might be the GPU/TPU, so the next one is IO. We are a system company, our product is the full system, as we control all aspects of the system. We have a 1.2 Tb/s ethernet interconnect to feed the engine to keep up with the compute

06:34PM EDT - Q: How long does it take a compile a model over 400k units? A: It's an algorithmically complex search space problem. Annealing and heuristics bring that down - we are borrowing many ideas from the EDA industry. Our problem is simpler than billions of LEs on FPGAs, so we're in the minutes.

06:34PM EDT - That's a wrap. Next talk is 4096 RISC-V chip

POST A COMMENT

13 Comments

View All Comments

  • jamessw - Monday, September 21, 2020 - link

    I will definitely share your post on my https://www.bestessays.com/edu_college.php website. My audience will be happy to find this info. Thanks! Reply
  • Homework Help - Monday, September 28, 2020 - link

    If you have any problem with writing your assignment, you don’t have to worry because we understand the importance of assignments in your academics. We understand how important it is for you to get good grades/marks in your assignments. Friends, you are just a click away from getting better grades in your assignment. Our assignment help service with its experienced and expert provide all sorts of assignment help to the students. Our assignment help services are affordable and you can get the maximum value for your money.
    Our services are available 24/7. Talk to us any time. We guarantee you quality and 100% satisfaction.

    Visit us Now !!!******************
    <a href="http://helpinhomework.org/blog/how-to-write-an-ess... to write a good essay</a>
    Reply
  • Alex2131 - Thursday, November 26, 2020 - link

    Abdulla Thani Techical Services L.L.C ranks high amongst < a href=''http://technicalservice.ae/services/fitout-work/ "> list of interior fitout companies in Dubai </a>. Our well-experienced team of professionals has the skills to give you the best Fit-out services. By making use of high-quality material and available resources we are guaranteed to provide you the most satisfactory results. Reply
  • Integral23 - Thursday, November 26, 2020 - link

    A leading private investigation company, Lipstick Investigations famous for its investigation services. We have a wealth of experience in matters including cheating partners and infidelity investigations. We specialize in family law investigations, find my logical mother, and other missing person investigations.
    https://lipstickinvestigations.com.au/company-prof...
    Reply
  • Integral23 - Thursday, November 26, 2020 - link

    Integral Investigations offers a range of different investigation services. Our operators can provide comprehensive services for: Missing Persons, Surveillance, De-bugging, Infidelity, <a href="https://integralinvestigations.com.au/services/&qu... Follow my partner</a> , Background and Asset Checks, Family Law and Custody Investigations, Forensic Telephone and Computer Investigations, Undercover, Fraud, Liability, Criminal, and Unsolved Cases. Reply
  • Integral23 - Thursday, November 26, 2020 - link

    Integral Investigations offers a range of different investigation services. Our operators can provide comprehensive services for: Missing Persons, Surveillance, De-bugging, Infidelity, Follow my partner, Background and Asset Checks, Family Law and Custody Investigations, Forensic Telephone and Computer Investigations, Undercover, Fraud, Liability, Criminal, and Unsolved Cases. They also have a website https://integralinvestigations.com.au/services/ Reply
  • Goodasnew - Saturday, December 12, 2020 - link

    Good As New Pressure Washing is a family owned business that specializes in Exterior Surface Cleaning. We are based in Harris County GA, but our service area includes Harris County and the entire surrounding area. We virtually cleans and remove rust stains from vinyl siding. We offer free estimate.
    Website: https://www.goodasnewpressurewashingllc.com/reside...
    Reply
  • Integral investigations - Wednesday, December 30, 2020 - link

    Integral investigation Sydney offers different services for your case that are performed by private detectives.<a href="https://integralinvestigations.com.au/private-dete... detective Sydney's cost </a> is carrying out all types of private also business investigations charge you a flat rate per hour with everything else included. Our industry offers you private investigation throughout Australia. The thing to be proud of is we are a discreet and affordable private investigation industry in Australia. Reply
  • Technical Service - Wednesday, December 30, 2020 - link

    How to decor and renovate the house through <a href="https://technicalservice.ae/carpentry-work-service... carpentry work services in Dubai </a>?This desire can now be easily fulfilled by ABDULLA THANI’S GROUP. ABDULLA THANI’S GROUP are providing carpentry work services in Dubai by name of TECHNICAL SERVICES BY ABDULLA THANI’S GROUP. Reply
  • Technical investigation - Thursday, December 31, 2020 - link

    Dubai is one of the most attractive cities in the world due to its catchy infrastructure and well-known buildings. We also have a <a href="https://technicalservice.ae/wall-partition-and-fal... False Ceiling Contractor in Dubai</a> for commercial and residential spaces as Beautiful and ethical Commercial and residential layout of houses is highly demanded in Dubai. In this regard, Abdullah Thani’s group of technical works are providing high-quality services to their clients. Abdullah Thani group provides many services. Reply

Log in

Don't have an account? Sign up now