СÐò

AIGC£¨AI-Generated Content£¬£¬£¬£¬£¬È˹¤ÖÇÄÜÉú²úÄÚÈÝ£©½üÆÚÉú³¤Ñ¸ÃÍ£¬£¬£¬£¬£¬µü´úËÙÂʸüÊÇ·ºÆðÖ¸Êý¼¶µÄ±¬·¢Ê½ÔöÌí¡£¡£¡£¡£¡£¡£¡£ÆäÖУ¬£¬£¬£¬£¬GPT-4ºÍÎÄÐÄÒ»ÑÔµÄÍƳöÒýÆðÁËÈËÃǶÔÆäÉÌÒµ¼ÛÖµºÍÓ¦Óó¡¾°µÄ¸ß¶È¹Ø×¢¡£¡£¡£¡£¡£¡£¡£Ëæ×ÅAIGCµÄÉú³¤£¬£¬£¬£¬£¬ÑµÁ·Ä£×Ó²ÎÊý¹æÄ£´ÓǧÒÚµ½ÍòÒÚ¼¶±ð£¬£¬£¬£¬£¬µ×²ãGPUÖ§³Ö¹æÄ£Ò²µÖ´ïÁËÍò¿¨¼¶±ð¡£¡£¡£¡£¡£¡£¡£Óɴ˵¼ÖµÄÍøÂç¹æÄ£Ò»Ö±Ôö´ó£¬£¬£¬£¬£¬ÍøÂç½Úµã¼äͨѶÃæÁÙ×ÅÔ½À´Ô½´óµÄÌôÕ½¡£¡£¡£¡£¡£¡£¡£ÔÚ´ËÅä¾°Ï£¬£¬£¬£¬£¬ÔõÑùÌáÉýAI·þÎñÆ÷ÅÌËãÄÜÁ¦ºÍ×éÍøͨѶÄÜÁ¦²¢¼æ¹Ë±¾Ç®£¬£¬£¬£¬£¬ÒѳÉΪĿ½ñÈ˹¤ÖÇÄÜÁìÓòµÄÖ÷ÒªÑо¿Æ«ÏòÖ®Ò»¡£¡£¡£¡£¡£¡£¡£

stake¹ÙÍøÍøÂçÕë¶ÔAIGCËãÁ¦¡¢GPUʹÓÃÂÊÓëÍøÂçµÄ¹Øϵ£¬£¬£¬£¬£¬ÒÔ¼°Ö÷Á÷HPC×éÍøÃæÁÙµÄÌôÕ½£¬£¬£¬£¬£¬ÍƳöÁËÒµ½çÏȽøµÄ“ÖÇËÙ”DDC£¨Distributed Disaggregated Chassis£¬£¬£¬£¬£¬ÂþÑÜʽÊèɢʽ»úÏ䣩¸ßÐÔÄÜÍøÂç¼Æ»®£¬£¬£¬£¬£¬ÎªAIGCÓªÒµÂòͨ“È潶þÂö”£¬£¬£¬£¬£¬ÖúÁ¦ËãÁ¦Í»·ÉÃͽø¡£¡£¡£¡£¡£¡£¡£

stake¹ÙÍøÍøÂçDDC²úÆ·ÅþÁ¬·½·¨Ê¾Òâͼ

AIGCËãÁ¦¡¢GPUʹÓÃÂÊÓëÍøÂçµÄ¹Øϵ

ChatGPTµÄѵÁ·Ê±¼äºÍGPUʹÓÃÂʵĹØϵ

ÒÔChatGPTΪÀý£¬£¬£¬£¬£¬ÔÚËãÁ¦·½Ã棬£¬£¬£¬£¬Ê¹ÓÃ΢ÈíAzure AI³¬Ëã»ù´¡ÉèÊ©£¨ÓÉ10000¿é V100 GPU×é³ÉµÄ¸ß´ø¿í¼¯Èº£©ÉϾÙÐÐѵÁ·£¬£¬£¬£¬£¬×ÜËãÁ¦ÏûºÄÔ¼3640PF-days£¨¼´Ã¿ÃëÒ»ÍòÍòÒÚ´ÎÅÌË㣬£¬£¬£¬£¬ÔËÐÐ3640Ì죩£¬£¬£¬£¬£¬ÕâÀï×ö¸ö¹«Ê½»»ËãÒ»ÏÂ10000¿éV100ÐèҪѵÁ·¶à¾Ã£º

ChatGPTËãÁ¦ºÍѵÁ·Ê±¼ä±í

×¢£ºChatGPTËãÁ¦ÐèÇóΪÍøÉÏ»ñÈ¡£¬£¬£¬£¬£¬Ôڴ˽ö¹©²Î¿¼¡£¡£¡£¡£¡£¡£¡£OpenAI ÔÚËûÃǵÄÎÄÕ“AI and Compute”ÖмÙÉèʹÓÃÂÊΪ 33%¡£¡£¡£¡£¡£¡£¡£NVIDIA¡¢Ë¹Ì¹¸£ºÍ΢ÈíµÄÒ»×éÑо¿Ö°Ô±ÔÚÂþÑÜʽϵͳÉÏѵÁ·´óÐÍÓïÑÔÄ£×ÓµÄʹÓÃÂʵִïÁË 44% µ½ 52%¡£¡£¡£¡£¡£¡£¡£

ChatGPT¹ØÓÚѵÁ·Ê±¼äµÄ»Ø¸²

ƾ֤ChatGPTµÄ»Ø¸´À´¿´£¬£¬£¬£¬£¬½ÏÁ¿ÇкÏÉÏÃæ±í¸ñÅÌËã³öÀ´µÄʱ¼ä£¬£¬£¬£¬£¬Ê¹ÓÃÂÊÓ¦¸Ã»áÔÚ50%×óÓÒ¡£¡£¡£¡£¡£¡£¡£

¿ÉÒÔ¿´³öÓ°ÏìÒ»¸öÄ£×ÓµÄѵÁ·Ê±³¤Ö÷ÒªÒòËØÔÚÓÚGPUµÄʹÓÃÂÊ£¬£¬£¬£¬£¬ÒÔ¼°GPU¼¯Èº´¦Öóͷ£ÄÜÁ¦¡£¡£¡£¡£¡£¡£¡£¶øÕâЩҪº¦Ö¸±êÓÖÓëÍøÂçЧÂÊÇ×½üÏà¹Ø¡£¡£¡£¡£¡£¡£¡£ÍøÂçЧÂÊÊÇÓ°ÏìAI¼¯ÈºÖÐGPUʹÓÃÂʵÄÒ»¸öÖ÷ÒªÒòËØ¡£¡£¡£¡£¡£¡£¡£ÔÚAI¼¯ÈºÖУ¬£¬£¬£¬£¬GPUͨ³£ÊÇÅÌËã½ÚµãµÄ½¹µã×ÊÔ´£¬£¬£¬£¬£¬ÓÉÓÚËüÃÇ¿ÉÒÔ¸ßЧµØ´¦Öóͷ£´ó¹æÄ£µÄÉî¶ÈѧϰʹÃü¡£¡£¡£¡£¡£¡£¡£È»¶ø£¬£¬£¬£¬£¬GPUµÄʹÓÃÂÊÊܵ½¶à¸öÒòËصÄÓ°Ï죬£¬£¬£¬£¬ÆäÖÐÍøÂçЧÂÊÊÇÒ»¸öÒªº¦ÒòËØ¡£¡£¡£¡£¡£¡£¡£

ÍøÂçЧÂÊÓëGPUʹÓÃÂʵĹØϵ

ÍøÂçÔÚAIѵÁ·ÖÐÊÎÑÝ×ÅÖÁ¹ØÖ÷ÒªµÄ½ÇÉ«¡£¡£¡£¡£¡£¡£¡£AI¼¯ÈºÍ¨³£Óɶà¸öÅÌËã½ÚµãºÍ´æ´¢½Úµã×é³É£¬£¬£¬£¬£¬ÕâЩ½ÚµãÐèҪƵÈԵؾÙÐÐͨѶºÍÊý¾Ý½»Á÷¡£¡£¡£¡£¡£¡£¡£ÈôÊÇÍøÂçЧÂʵÍÏ£¬£¬£¬£¬£¬ÕâЩ½ÚµãÖ®¼äµÄͨѶ½«»á±äµÃ»ºÂý£¬£¬£¬£¬£¬Õ⽫ֱ½ÓÓ°Ïìµ½AI¼¯ÈºµÄËãÁ¦¡£¡£¡£¡£¡£¡£¡£

µÍЧµÄÍøÂç¿ÉÄܵ¼ÖÂÒÔÏÂÎÊÌ⣬£¬£¬£¬£¬´Ó¶ø½µµÍGPUʹÓÃÂÊ£º

Êý¾Ý´«Êäʱ¼äÔöÌí£ºÔÚµÍЧµÄÍøÂçÖУ¬£¬£¬£¬£¬Êý¾Ý´«ÊäµÄʱ¼ä½«»áÔöÌí¡£¡£¡£¡£¡£¡£¡£µ±GPUÐèÒªÆÚ´ýÊý¾Ý´«ÊäÍê³Éºó²Å»ª¾ÙÐÐÅÌËãʱ£¬£¬£¬£¬£¬GPUʹÓÃÂʽ«»á½µµÍ£»£»£»£»£»

ÍøÂç´ø¿íÆ¿¾±£ºÔÚAI¼¯ÈºÖУ¬£¬£¬£¬£¬GPUͨ³£ÐèҪƵÈÔµØÓëÆäËûÅÌËã½Úµã¾ÙÐÐÊý¾Ý½»Á÷¡£¡£¡£¡£¡£¡£¡£ÈôÊÇÍøÂç´ø¿íȱ·¦£¬£¬£¬£¬£¬GPU½«ÎÞ·¨»ñµÃ×ã¹»µÄÊý¾Ý¾ÙÐÐÅÌË㣬£¬£¬£¬£¬´Ó¶øµ¼ÖÂGPUʹÓÃÂʽµµÍ£»£»£»£»£»

ʹÃüµ÷Àí²»Æ½ºâ£ºÔÚµÍЧµÄÍøÂçÖУ¬£¬£¬£¬£¬Ê¹Ãü¿ÉÄܻᱻ·ÖÅɵ½ÓëGPU²î±ðµÄÅÌËã½ÚµãÉÏ¡£¡£¡£¡£¡£¡£¡£µ±ÐèÒª´ó×ÚµÄÊý¾Ý´«Êäʱ£¬£¬£¬£¬£¬Õâ¿ÉÄܻᵼÖÂGPUÏÐÖÃÆÚ´ý£¬£¬£¬£¬£¬´Ó¶ø½µµÍGPUʹÓÃÂÊ¡£¡£¡£¡£¡£¡£¡£

ΪÁËÌá¸ßGPUʹÓÃÂÊ£¬£¬£¬£¬£¬ÐèÒªÓÅ»¯ÍøÂçЧÂÊ¡£¡£¡£¡£¡£¡£¡£Õâ¿ÉÒÔͨ¹ý½ÓÄɸü¿ìµÄÍøÂçÊÖÒÕ¡¢ÓÅ»¯ÍøÂçÍØÆ˽ṹ¡¢ºÏÀíÉèÖôø¿íµÈÒªÁìÀ´ÊµÏÖ¡£¡£¡£¡£¡£¡£¡£ÔÚѵÁ·Ä£×ÓÖУ¬£¬£¬£¬£¬ÂþÑÜʽѵÁ·µÄ²¢ÐжȣºÊý¾Ý²¢ÐС¢ÕÅÁ¿²¢ÐÐÓëÁ÷Ë®²¢ÐоöÒéÁËGPU´¦Öóͷ£µÄÊý¾ÝÖ®¼äµÄͨѶģ×Ó¡£¡£¡£¡£¡£¡£¡£Ä£×ÓÖ®¼äµÄͨѶЧÂÊÊܵ½ÒÔϼ¸¸öÒòËصÄÓ°Ï죺

Ó°ÏìͨѶµÄÒòËØ

ÆäÖУ¬£¬£¬£¬£¬´ø¿íºÍ×°±¸×ª·¢Ê±ÑÓÊܵ½Ó²¼þÏÞÖÆ£¬£¬£¬£¬£¬¶Ë´¦Öóͷ£Ê±ÑÓÊÜÊÖÒÕÑ¡Ôñ£¨TCP or RDMA£©Ó°Ï죬£¬£¬£¬£¬RDMA»á¸üµÍ£¬£¬£¬£¬£¬ÅŶӺÍÖØ´«ÔòÊܵ½ÍøÂçÓÅ»¯ºÍÊÖÒÕÑ¡ÔñµÄÓ°Ïì¡£¡£¡£¡£¡£¡£¡£

ƾ֤Á¿»¯Ä£×Ó[1]£ºGPUʹÓÃÂÊ = GPUÄÚµü´úÅÌËãʱ¼ä/£¨GPUÄÚµü´úÅÌËãʱ¼ä+ÍøÂç×ÜÌåͨѶʱ¼ä£©À´ÅÌËãµÃ³öÒÔϽáÂÛ£º

´ø¿íÍÌÍÂÓëGPUʹÓÃÂʵÄÇúÏßͼ                                  ¶¯Ì¬Ê±ÑÓºÍGPUʹÓÃÂʵÄÇúÏßͼ

¿ÉÒÔ¿´µ½ÍøÂç´ø¿íÍÌÍ¡¢¶¯Ì¬Ê±ÑÓ£¨ÓµÈû/¶ª°ü£©¶ÔGPUʹÓÃÂÊÓ°ÏìÏÔ×Å¡£¡£¡£¡£¡£¡£¡£

ƾ֤ͨѶ×ÜʱÑÓµÄ×é³ÉÀ´¿´£º

ͨѶ×ÜʱÑÓ×é³Éͼ

¾²Ì¬Ê±ÑÓÏà½ÏÖ®ÏÂÓ°Ïì¸üС£¬£¬£¬£¬£¬ÒÔÊǸüÓ¦¸Ã×ÅÖØȥ˼Á¿ÔõÑùïÔÌ­¶¯Ì¬Ê±ÑÓ£¬£¬£¬£¬£¬ÕâÑù¿ÉÒÔÓÐÓõÄÌáÉýGPUµÄʹÓÃÂÊ£¬£¬£¬£¬£¬´Ó¶øµÖ´ïÌáÉýËãÁ¦µÄÄ¿µÄ¡£¡£¡£¡£¡£¡£¡£

Ö÷Á÷HPC×éÍøÃæÁÙµÄÌôÕ½

IB×éÍøÌÚ¹óÇҹرÕ

Infiniband×éÍøÊÇÄ¿½ñ¸ßÐÔÄÜÍøÂçµÄЧ¹û×îӎ⣬£¬£¬£¬£¬Ê¹Óó¬¸ß´ø¿íºÍ»ùÓÚCreditµÄ»úÖÆÈ·±£ÎÞÓµÈûºÍ³¬µÍʱÑÓ£¬£¬£¬£¬£¬¿ÉÊÇÒ²ÊÇ×îÌÚ¹óµÄ½â·¨£¬£¬£¬£¬£¬Ïà±Èͬ´ø¿íϹŰåÒÔÌ«ÍøµÄ×éÍø»á¹óÊý±¶¡£¡£¡£¡£¡£¡£¡£Í¬Ê±InfinibandÊÖÒչرÕ£¬£¬£¬£¬£¬ÒµÄÚÏÖÔÚ³ÉÊ칩ӦÉ̽ö1¼Ò£¬£¬£¬£¬£¬¹ØÓÚ×îÖÕÓû§À´Ëµ£¬£¬£¬£¬£¬ÎÞ·¨ÊµÏÖµÚ¶þ»õÔ´¡£¡£¡£¡£¡£¡£¡£

ÒÔÊÇÒµÄÚ´ó´ó¶¼Óû§»áÑ¡Ôñ¹Å°åÒÔÌ«Íø×éÍøµÄ¼Æ»®¡£¡£¡£¡£¡£¡£¡£

PFCºÍECN¿ÉÄÜ´¥·¢½µËÙ

Ä¿½ñ¸ßÐÔÄÜÍøÂçÖ÷Á÷×éÍø¼Æ»®ÊÇ»ùÓÚRoCE v2À´×齨֧³ÖRDMAµÄÍøÂç¡£¡£¡£¡£¡£¡£¡£ÆäÖÐÖ÷ÒªµÄÁ½Ïî´îÅäÊÖÒÕÊÇPFCºÍECN£¬£¬£¬£¬£¬Á½Õß¾ùÊÇΪÁË×èÖ¹Á´Â·ÖеÄÓµÈû¶ø±¬·¢µÄÊÖÒÕ¡£¡£¡£¡£¡£¡£¡£

¶à¼¶PFC×éÍøÏ»áÕë¶Ô½»Á÷»úÈë¿Ú£¨Ingress£©ÓµÈû£¬£¬£¬£¬£¬Ö𼶷´Ñ¹µ½Ô´¶Ë·þÎñÆ÷ÔÝÍ£·¢ËÍ£¬£¬£¬£¬£¬»º½âÍøÂçÓµÈû£¬£¬£¬£¬£¬¹æ±Ü¶ª°ü£»£»£»£»£»µ«¸Ã¼Æ»®Ôڶ༶×éÍøÏ¿ÉÄÜÅöÃæÁÙPFC Deadlockµ¼ÖÂRDMAÁ÷Á¿×èֹת·¢µÄΣº¦¡£¡£¡£¡£¡£¡£¡£

ͼƬ

PFCÊÂÇé»úÖÆʾÒâͼ

¶øECNÔò»á»ùÓÚ¶Ô½»Á÷»ú³ö¿Ú£¨Egress£©ÓµÈûµÄÄ¿µÄ¶Ë¸ÐÖª£¬£¬£¬£¬£¬Ö±½ÓÌìÉúÒ»¸öRoCEv2 CNP°ü֪ͨԴ¶Ë½µËÙ£¬£¬£¬£¬£¬Ô´·þÎñÆ÷ÊÕµ½CNP±¨ÎÄ£¬£¬£¬£¬£¬¾«×¼½µµÍ¶ÔÓ¦QPµÄ·¢ËÍËÙÂÊ£¬£¬£¬£¬£¬»º½âÓµÈûµÄͬʱ×èÖ¹ÎÞ²î±ð½µËÙ¡£¡£¡£¡£¡£¡£¡£

ECN±ê¼ÇλʾÒâͼ

ÕâÁ½ÏîÊÖÒÕ×Ô¼º²¢Ã»ÓÐʲôÎÊÌ⣬£¬£¬£¬£¬¶¼ÊÇΪÏàʶ¾öÓµÈû¶ø½µÉúµÄÊÖÒÕ£¬£¬£¬£¬£¬¿ÉÊǽÓÄÉÕâÖÖÊÖÒÕºó¿ÉÄܻᱻÍøÂçÖпÉÄܱ¬·¢µÄÓµÈû¶øƵÈÔ´¥·¢£¬£¬£¬£¬£¬×îÖջᵼÖÂÔ´¶ËÔÝÍ£»£»£»£»£»ò½µËÙ·¢ËÍ£¬£¬£¬£¬£¬Í¨Ñ¶´ø¿í»á½µµÍ£¬£¬£¬£¬£¬»á¶ÔGPUʹÓÃÂʱ¬·¢½ÏÁ¿´óµÄÓ°Ï죬£¬£¬£¬£¬´Ó¶øÔì³ÉÕû¸ö¸ßÐÔÄÜÍøÂçµÄËãÁ¦±»À­µÍ¡£¡£¡£¡£¡£¡£¡£

ECMP²»Æ½ºâ¿ÉÄܻᵼÖÂÓµÈû

ÔÚAIѵÁ·ÅÌËãÖлáÓÐAll-ReduceºÍAll-to-AllÁ½ÖÖÖ÷ÒªµÄÄ£×Ó£¬£¬£¬£¬£¬Á½ÖÖÄ£×Ó¶¼ÐèҪƵÈԵĴÓÒ»¸öGPUµ½ÁíÍâ¶à¸öGPU¾ÙÐÐͨѶ¡£¡£¡£¡£¡£¡£¡£

All-to-AllÄ£×Ó                       All-ReduceÄ£×Ó

ÔڹŰå×éÍøÏ£¬£¬£¬£¬£¬ToRºÍLeaf×°±¸½ÓÄÉ·ÓÉ+ECMPµÄ×éÍøģʽ£¬£¬£¬£¬£¬ECMP»á»ùÓÚÁ÷¾ÙÐйþÏ£¸ºÔØѡ·£¬£¬£¬£¬£¬ÓÐÒ»ÖÖ¼«¶ËÇéÐξÍÊÇijһÌõECMPÁ´Â·ÓÉÓÚÒ»Ìõ´óÏóÁ÷¶øÅÜÂú£¬£¬£¬£¬£¬ÆäÓà¶àÌõECMPÁ´Â·Ïà¶Ô¿ÕÏУ¬£¬£¬£¬£¬Ôì³É¸ºÔز»¾ùµÄÇéÐΡ£¡£¡£¡£¡£¡£¡£

¹Å°åECMP°²ÅÅͼ

ÔÚÄÚ²¿Ä£Äâ8ÌõECMPÁ´Â·µÄ²âÊÔÇéÐÎÏ£¬£¬£¬£¬£¬²âÊÔЧ¹ûÈçÏ£º

ECMPÁ÷Á¿²âÊÔЧ¹û

¿ÉÒÔ¿´³ö£¬£¬£¬£¬£¬»ùÓÚÁ÷µÄECMP»áÔì³É½ÏÏÔ×ŵÄij¼¸ÌõÁ´Â·Õ¼Óã¨ECMP1-5ºÍ1-6£©ºÍ¿ÕÏУ¨ECMP1-0ÖÁ1-3½Ï¿ÕÏУ©£¬£¬£¬£¬£¬¶øÔÚAll-ReduceºÍAll-to-AllµÄÁ½ÖÖÄ£×ÓÏ£¬£¬£¬£¬£¬ ¾ÍºÜÈÝÒ×Ôì³ÉÒ»Ìõõ辶ÓÉÓÚECMPµÄ¸ºÔز»¾ù¶øÓµÈû£¬£¬£¬£¬£¬Ò»µ©ÓµÈûÔì³ÉÖØ´«£¬£¬£¬£¬£¬¾Í»áÌáÉý×ÜÌåµÄͨѶ×ÜʱÑÓ£¬£¬£¬£¬£¬´Ó¶ø½µµÍGPUʹÓÃÂÊ¡£¡£¡£¡£¡£¡£¡£

ÒÔÊÇ£¬£¬£¬£¬£¬ÎªÏàʶ¾ö´ËÀàÎÊÌ⣬£¬£¬£¬£¬Ñо¿½çÌá³öÁËphost¡¢Homa¡¢NDP¡¢1RMA ºÍ AeolusµÈ¸»ºñµÄ½â¾ö¼Æ»®£¬£¬£¬£¬£¬ËüÃÇÔÚ²î±ðˮƽÉϽâ¾öÁË incast£¬£¬£¬£¬£¬ »¹½â¾öÁ˸ºÔØƽºâºÍµÍÑÓ³ÙÇëÇó/ÏìÓ¦Á÷Á¿µÄÎÊÌâ¡£¡£¡£¡£¡£¡£¡£¿ÉÊÇÒ²´øÀ´ÁËеÄÌôÕ½£¬£¬£¬£¬£¬ÍùÍùÕâЩÑо¿µÄ¼Æ»®¶¼ÊÇÐèÒª¶Ëµ½¶ËÀ´½â¾öÎÊÌ⣬£¬£¬£¬£¬¶ÔÖ÷»ú¡¢Íø¿¨¡¢ÍøÂçµÄ¸Ä¶¯½Ï´ó£¬£¬£¬£¬£¬¹ØÓÚÒ»Ñùƽ³£Óû§¶øÑÔ£¬£¬£¬£¬£¬±¾Ç®½Ï¸ß¡£¡£¡£¡£¡£¡£¡£

¿òʽ½»Á÷»ú×éAI¼¯ÈºµÄÌôÕ½

ÍâÑóÓв¿·Ö»¥ÁªÍø¹«Ë¾¼ÄÏ£ÍûÓÚʹÓýÓÄÉDNXоƬ֧³ÖVOQÊÖÒյĿòʽ½»Á÷»úÀ´½â¾ö¸ºÔز»Æ½ºâ´øÀ´µÄ´ø¿íʹÓÃÂʵ͵ÄÎÊÌ⣬£¬£¬£¬£¬µ«Ò²ÃæÁÙÒÔϼ¸¸öÌôÕ½¡£¡£¡£¡£¡£¡£¡£

À©Õ¹ÄÜÁ¦Ò»Ñùƽ³££¬£¬£¬£¬£¬»ú¿ò¾ÞϸÏÞÖÆÁË×î´ó¶Ë¿ÚÊý£¬£¬£¬£¬£¬ÈçÏë×ö¸ü´ó¹æÄ£µÄ¼¯Èº£¬£¬£¬£¬£¬ÐèÒªºáÏòÀ©Õ¹¶à¸ö»ú¿ò£¬£¬£¬£¬£¬Ò²»á±¬·¢¶à¼¶PFCºÍECMPµÄÁ´Â·£¬£¬£¬£¬£¬ÒÔÊÇ¿òÖ»ÊʺÏÓÚС¹æÄ£°²ÅÅ£»£»£»£»£»

×°±¸¹¦ºÄ´ó£¬£¬£¬£¬£¬»ú¿òÄÚÏß¿¨Ð¾Æ¬¡¢FabricоƬ¡¢µçÉȵÈÊýÄ¿Öڶ࣬£¬£¬£¬£¬µ¥×°±¸µÄ¹¦ºÄ¼«´ó£¬£¬£¬£¬£¬ÇáËÉÁè¼Ý2ÍòÍߣ¬£¬£¬£¬£¬ÓеÄÉõÖÁ3Íò¶àÍߣ¬£¬£¬£¬£¬¶Ô»ú¹ñµçÁ¦ÒªÇó¸ß£»£»£»£»£»

µ¥×°±¸¶Ë¿ÚÊýÄ¿¶à£¬£¬£¬£¬£¬¹ÊÕÏÓò´ó¡£¡£¡£¡£¡£¡£¡£

ÒÔÊÇ»ùÓÚÒÔÉÏÔµ¹ÊÔ­ÓÉ£¬£¬£¬£¬£¬¿òʽװ±¸Ö»ÊʺÏС¹æÄ£°²ÅÅAIÅÌË㼯Ⱥ¡£¡£¡£¡£¡£¡£¡£

ÐÂÐÎ̬DDC²úÆ·½µÉú£¬£¬£¬£¬£¬Ö§³ÖAIGC¸ßÐÔÄÜÍøÂç

DDCÊÇÒ»ÖÖÂþÑÜʽ½âñî»ú¿ò×°±¸µÄ½â¾ö¼Æ»®£¬£¬£¬£¬£¬½ÓÄɵÄоƬºÍÒªº¦ÊÖÒÕÓë¹Å°å¿òʽ½»Á÷»úÏÕЩÏàͬ£¬£¬£¬£¬£¬µ«DDC¼Ü¹¹¼òÆÓÖ§³Öµ¯ÐÔÀ©Õ¹ºÍ¹¦Ð§¿ìËÙµü´ú¡¢¸üÒ×°²ÅÅ¡¢µ¥»ú¹¦ºÄµÍ¡£¡£¡£¡£¡£¡£¡£

ÈçÏÂͼËùʾ£¬£¬£¬£¬£¬ÓªÒµÏß¿¨×÷Ϊǰ¶Ë³ÉΪNCP½ÇÉ«£¬£¬£¬£¬£¬½»Á÷Íø°å×÷Ϊºó¶Ë³ÉΪNCF½ÇÉ«£¬£¬£¬£¬£¬Ô­ÏÈÁ½ÕßÖ®¼äµÄÅþÁ¬Æ÷×é¼þÏÖÔÚ±»¹âÏËÏßÀÂÈ¡´ú£¬£¬£¬£¬£¬Ô­Óпòʽװ±¸µÄ¹ÜÀíÒýÇæÔÚDDC¼Ü¹¹ÖÐÒ²³ÉΪÁËNCC¼¯ÖÐ/ÂþÑÜʽµÄ¹ÜÀí×é¼þ¡£¡£¡£¡£¡£¡£¡£

DDC²úÆ·ÅþÁ¬·½·¨Ê¾Òâͼ

DDCÖ§³Ö³¬´ó¹æÄ£°²ÅÅ

DDC¼Ü¹¹Ïà½ÏÓÚ¿òʽ¼Ü¹¹µÄÓÅÊÆÔÚÓÚ¿ÉÒÔÌṩµ¯ÐÔ¿ÉÀ©Õ¹ÐÔ£¬£¬£¬£¬£¬×éÍø¹æÄ£¿£¿£¿£¿£¿ÉÒÔƾ֤AI¼¯Èº¾ÞϸÀ´ÎÞаѡÔñ¡£¡£¡£¡£¡£¡£¡£

µ¥POD×éÍøÖУ¬£¬£¬£¬£¬½ÓÄÉ96̨NCP×÷Ϊ½ÓÈ룬£¬£¬£¬£¬ÆäÖÐNCPÏÂÐй²36¸ö200G½Ó¿Ú£¬£¬£¬£¬£¬ÈÏÕæÅþÁ¬AIÅÌË㼯ȺµÄÍø¿¨¡£¡£¡£¡£¡£¡£¡£ÉÏÐй²40¸ö200G½Ó¿Ú×î´ó¿ÉÒÔÅþÁ¬40̨NCF£¬£¬£¬£¬£¬NCFÌṩ96¸ö200G½Ó¿Ú£¬£¬£¬£¬£¬¸Ã¹æÄ£ÉÏÏÂÐдø¿íΪ³¬ËÙ±È1.1:1¡£¡£¡£¡£¡£¡£¡£Õû¸öPOD¿ÉÖ§³Ö3456¸ö200GÍøÂç½Ó¿Ú£¬£¬£¬£¬£¬Æ¾Ö¤Ò»Ì¨·þÎñÆ÷Åä8¿éGPUÀ´ÅÌË㣬£¬£¬£¬£¬¿ÉÖ§³Ö432̨AIÅÌËã·þÎñÆ÷¡£¡£¡£¡£¡£¡£¡£

µ¥POD×éÍø¼Ü¹¹Í¼

¶à¼¶POD×éÍøÖУ¬£¬£¬£¬£¬¿ÉÒÔʵÏÖ»ùÓÚPODµÄ°´Ð轨Éè¡£¡£¡£¡£¡£¡£¡£ÓÉÓڸó¡¾°PODÖÐNCF×°±¸ÒªÎþÉüÒ»°ëµÄSerDesÓÃÓÚÅþÁ¬µÚ¶þ¼¶µÄNCF£¬£¬£¬£¬£¬ÒÔÊÇ´Ëʱµ¥POD½ÓÄÉ48̨NCP×÷Ϊ½ÓÈ룬£¬£¬£¬£¬ÏÂÐй²36¸ö200G½Ó¿Ú£¬£¬£¬£¬£¬µ¥PODÄÚ¿ÉÒÔÖ§³Ö1728¸ö200G½Ó¿Ú¡£¡£¡£¡£¡£¡£¡£Í¨¹ýºáÏòÔöÌíPODʵÏÖ¹æÄ£µÄÀ©ÈÝ£¬£¬£¬£¬£¬ÕûÌå×î´ó¿ÉÖ§³Ö10368¶à¸ö200GÍøÂç¶Ë¿Ú¡£¡£¡£¡£¡£¡£¡£

NCPÉÏÐÐ40¸ö200G½ÓPODÄÚ40̨NCF£¬£¬£¬£¬£¬PODÄÚNCF½ÓÄÉ48¸ö200G½Ó¿ÚÏÂÐУ¬£¬£¬£¬£¬48¸ö200G½Ó¿Ú·ÖΪ16¸öÒ»×éÉÏÐе½µÚ¶þ¼¶µÄNCF¡£¡£¡£¡£¡£¡£¡£µÚ¶þ¼¶NCF½ÓÄÉ40¸öƽÃ棬£¬£¬£¬£¬Ã¿¸öƽÃæ3̨µÄÉè¼Æ£¬£¬£¬£¬£¬»®·Ö¶ÔÓ¦ÔÚPODÄÚµÄ40̨NCF¡£¡£¡£¡£¡£¡£¡£

Õû¸öÍøÂçµÄPODÄÚʵÏÖÁ˳¬ËÙ±È1.1:1£¬£¬£¬£¬£¬¶øÔÚPODºÍ¶þ¼¶NCFÖ®¼äʵÏÖÁË1:1µÄÊÕÁ²±È¡£¡£¡£¡£¡£¡£¡£

200GµÄÍøÂç¶Ë¿Ú¼æÈÝ100GÍø¿¨½ÓÈ룬£¬£¬£¬£¬ÌØÊâÇéÐÎÏ¿ÉʹÓÃ1·Ö2»ò1·Ö4ÏßÀ¼æÈÝ25/50GÍø¿¨¡£¡£¡£¡£¡£¡£¡£

»ùÓÚVOQ+Cell»úÖƸºÔظüƽºâ£¬£¬£¬£¬£¬¶ª°üÂʸüµÍ

ÒÀÍзÖƬºóµÄCellsת·¢»úÖƾÙÐж¯Ì¬¸ºÔØƽºâ£¬£¬£¬£¬£¬ÊµÏÖÑÓ³ÙµÄÎȹÌÐÔ£¬£¬£¬£¬£¬½µµÍÁ˲î±ðÁ´Â·µÄ´ø¿í·åÖµ²î¡£¡£¡£¡£¡£¡£¡£

ת·¢Á÷³ÌÈçͼËùʾ£º

Ê×ÏÈ·¢ËͶ˴ÓÍøÂçÖÐÎüÊÕÊý¾Ý°ü²¢·ÖÀൽVOQsÖд洢£¬£¬£¬£¬£¬ÔÚ·¢ËÍÊý¾Ý°ü֮ǰ»áÏÈ·¢ËÍCredit±¨ÎÄÈ·¶¨ÎüÊÕ¶ËÊÇ·ñÓÐ×ã¹»µÄ»º´æ¿Õ¼ä´¦Öóͷ£ÕâЩ±¨ÎÄ£»£»£»£»£»

ÈôÊÇ¿ÉÒÔÔò½«Êý¾Ý°ü·ÖƬ³ÉCells²¢ÇÒ¶¯Ì¬¸ºÔØƽºâµ½ÖÐÐĵÄFabric½Úµã¡£¡£¡£¡£¡£¡£¡£ÕâЩCellsÔÚÎüÊն˻á¾ÙÐÐÖØ×éºÍ´æ´¢£¬£¬£¬£¬£¬½ø¶øת·¢µ½ÍøÂçÖС£¡£¡£¡£¡£¡£¡£

CellsÊÇ»ùÓÚÊý¾Ý°üµÄÇÐƬÊÖÒÕ£¬£¬£¬£¬£¬Ò»Ñùƽ³£¾ÞϸΪ 64-256Byte¡£¡£¡£¡£¡£¡£¡£

ÇÐƬºóµÄCellsƾ֤reachability table ÖÐ cell  destination µÄÅÌÎÊÀ´¾öÒéÔõÑùת·¢£¬£¬£¬£¬£¬²¢½ÓÄÉÂÖѯµÄ»úÖÆ·¢ËÍ¡£¡£¡£¡£¡£¡£¡£ÕâÑù×öµÄÀûÒæÏà±ÈECMP°´Á÷¾ÙÐйþÏ£ÅÌËãºóÑ¡ÔñijһÌõ·µÄģʽ£¬£¬£¬£¬£¬ÇÐƬºóµÄCells¸ºÔØ»á³ä·ÖʹÓõ½Ã¿Ò»ÌõÉÏÐÐÁ´Â·£¬£¬£¬£¬£¬ËùÓÐÉÏÐÐÁ´Â·µÄ´«ÊäÊý¾ÝÁ¿»á½üËÆÏàµÈ¡£¡£¡£¡£¡£¡£¡£

ÈôÊÇÎüÊÕ¶ËÔÝʱûÄÜÁ¦´¦Öóͷ£±¨ÎÄ£¬£¬£¬£¬£¬±¨ÎÄ»áÔÚ·¢ËͶ˵ÄVOQÖÐÔݴ棬£¬£¬£¬£¬²¢²»»áÖ±½Óת·¢µ½ÎüÊն˵¼Ö¶ª°üÎÊÌâµÄ±¬·¢£¬£¬£¬£¬£¬Ã¿Æ¬DNXоƬ¿ÉÒÔÌṩоƬÄÚOCB»º´æÒÔ¼°Æ¬Íâ8GBµÄHBM¸ßËÙ»º´æ£¬£¬£¬£¬£¬¶Ô200G¶Ë¿ÚÏ൱ÓÚ¿ÉÒÔ»º´æ150ms×óÓÒµÄÊý¾Ý¡£¡£¡£¡£¡£¡£¡£Ö»Óе±¶Ô¶ËCredit±¨ÎÄÃ÷È·¿ÉÒÔ½ÓÊÜʱ²Å»á·¢ËÍ¡£¡£¡£¡£¡£¡£¡£ÕâÑùµÄ»úÖÆÏ£¬£¬£¬£¬£¬³ä·ÖʹÓûº´æ¿ÉÒÔ´ó·ù¶ÈïÔÌ­¶ª°ü£¬£¬£¬£¬£¬ÉõÖÁ²»»á±¬·¢¶ª°üÇéÐΡ£¡£¡£¡£¡£¡£¡£ïÔÌ­Êý¾ÝÖØ´«£¬£¬£¬£¬£¬ÕûÌåͨѶʱÑÓ¸üÎȹ̸üµÍ£¬£¬£¬£¬£¬´Ó¶ø¿ÉÒÔÌá¸ß´ø¿íʹÓÃÂÊ£¬£¬£¬£¬£¬½ø¶øÌáÉýÓªÒµÍÌÍÂЧÂÊ¡£¡£¡£¡£¡£¡£¡£

PFCµ¥Ìø°²ÅÅϲ»»á±¬·¢ËÀËø

ƾ֤DDCµÄÂß¼­À´¿´£¬£¬£¬£¬£¬ËùÓÐNCPºÍNCF¿ÉÒÔ¿´³Éһ̨װ±¸£¬£¬£¬£¬£¬ÒÔÊÇÔÚ´ËÍøÂçÖа²ÅÅRDMAÓòºó£¬£¬£¬£¬£¬Ö»ÔÚÕë¶Ô·þÎñÆ÷µÄ½Ó¿Ú´¦±£´æ1¼¶µÄPFC£¬£¬£¬£¬£¬²»»áÏñ¹Å°åÍøÂçÒ»Ñù±¬·¢¶à¼¶PFCµÄѹÖÆÓëËÀËø¡£¡£¡£¡£¡£¡£¡£ÁíÍâƾ֤DDCµÄÊý¾Ýת·¢»úÖÆ£¬£¬£¬£¬£¬¿ÉÔÚ½Ó¿Ú´¦°²ÅÅECN£¬£¬£¬£¬£¬Ò»µ©ÔÚÄÚ²¿µÄCredit»ººÍ´æ»úÖÆÎÞ·¨Ö§³ÖÍ»·¢Á÷Á¿£¬£¬£¬£¬£¬¿ÉÒÔÏò·þÎñÆ÷¶Ë·¢ËÍCNP±¨ÎÄÒªÇó½µËÙ£¨Í¨³£ÇéÐÎÏÂÔÚAIµÄͨѶģ×ÓÏ£¬£¬£¬£¬£¬All-to-AllºÍAll-Reduce+CellÇÐƬ¿ÉÒÔ½«Á÷Á¿¾¡¿ÉÄܵÄƽºâ£¬£¬£¬£¬£¬ºÜÄÑ·ºÆð1¸ö¶Ë¿Ú±»´òÂúµÄÇéÐΣ¬£¬£¬£¬£¬ÒÔÊÇECNÔÚ´ó¶¼ÇéÐοÉÒÔ²»ÉèÖã©¡£¡£¡£¡£¡£¡£¡£

ÎÞNCCÉè¼Æ£¬£¬£¬£¬£¬½ÓÄÉÂþÑÜʽOSÌáÉý¿É¿¿ÐÔ

ÔÚ¹ÜÀí¿ØÖÆƽÃæÉÏ£¬£¬£¬£¬£¬ÎªÏàʶ¾ö¹ÜÀíÍø¹ÊÕÏÒÔ¼°NCCµ¥µã¹ÊÕϵÄÓ°Ï죬£¬£¬£¬£¬ÎÒÃÇ×÷·ÏÁËNCCµÄ¼¯ÖпØÖÆÃ棬£¬£¬£¬£¬¹¹½¨ÁËÂþÑÜʽOS£¬£¬£¬£¬£¬Í¨¹ýSDNÔËά¿ØÖÆÆ÷ͨ¹ý±ê×¼½Ó¿Ú£¨Netconf¡¢GRPCµÈ£©ÉèÖùÜÀí×°±¸£¬£¬£¬£¬£¬Ã¿Ì¨NCPºÍNCF×ÔÁ¦¹ÜÀí£¬£¬£¬£¬£¬ÓÐ×ÔÁ¦µÄ¿ØÖÆÃæºÍ¹ÜÀíÃæ¡£¡£¡£¡£¡£¡£¡£

²âÊÔ±ÈÕÕЧ¹û

´Ó¼Æ»®ÀíÂÛÉÏ˵£¬£¬£¬£¬£¬DDCÓµÓÐÖ§³Öµ¯ÐÔÀ©Õ¹ºÍ¹¦Ð§¿ìËÙµü´ú¡¢¸üÒ×°²ÅÅ¡¢µ¥»ú¹¦ºÄµÍµÈÖÚ¶àÓÅÊÆ£»£»£»£»£»µ«´ÓÏÖʵ½Ç¶È³ö·¢£¬£¬£¬£¬£¬¹Å°å×éÍøÒ²ÓµÓÐÖîÈçÊÐÃæ¿ÉÑ¡Æ·ÅƺͲúÆ·õ辶½Ï¶à¡¢¿ÉÖ§³Ö¸ü´ó¹æÄ£µÄ¼¯ÈºµÈÊÖÒÕ³ÉÊì´øÀ´µÄÓÅÊÆ¡£¡£¡£¡£¡£¡£¡£Òò´ËÔÚ¿Í»§ÃæÁÙÏîÄ¿ÐèÇóʱÊÂʵÊÇÑ¡Ôñ¸ü¸ßÐÔÄܵÄDDC£¬£¬£¬£¬£¬Õվɸü´ó¹æÄ£°²ÅŵĹŰå×éÍø£¬£¬£¬£¬£¬¿ÉÒԲο¼ÏÂÃæµÄ±ÈÕÕ¼°²âÊÔЧ¹û£º

¹Å°å×éÍøÓëDDC²âÊÔ±ÈÕÕЧ¹ûͼ

ͬʱÎÒÃÇʹÓÃOpenMPI²âÊÔÌ×¼þ¾ÙÐÐÁË¿òʽװ±¸£¨¿òʽװ±¸ºÍDDCÔ­ÀíÏàͬ£¬£¬£¬£¬£¬±¾´Î½ÓÄÉ¿òʽ²âÊÔ£©ºÍ¹Å°å×éÍø×°±¸µÄ±ÈÕÕÄ£Äâ²âÊÔ£¬£¬£¬£¬£¬½áÂÛÊÇÔÚAll-to-All³¡¾°Ï£¬£¬£¬£¬£¬Ïà½ÏÓڹŰåµÄ×éÍø£¬£¬£¬£¬£¬¿òʽװ±¸´ø¿íʹÓÃÂÊÌáÉýÔ¼20%£¨¶ÔÓ¦GPUʹÓÃÂÊÌáÉý8%×óÓÒ£©¡£¡£¡£¡£¡£¡£¡£

¿òʽװ±¸ºÍ¹Å°å×éÍø×°±¸µÄ±ÈÕÕÄ£Äâ²âÊÔ

stake¹ÙÍø×°±¸ÏÈÈÝ

»ùÓÚ¶Ô¿Í»§ÐèÇóµÄÉî¿ÌÃ÷È·£¬£¬£¬£¬£¬stake¹ÙÍøÍøÂçÒѾ­ÂÊÏÈÍƳöÁËÁ½¿î¿É½»¸¶²úÆ·£¬£¬£¬£¬£¬»®·ÖÊÇ200G NCP½»Á÷»úºÍ200G NCF½»Á÷»ú¡£¡£¡£¡£¡£¡£¡£

NCP£ºRG-S6930-36DC40F1½»Á÷»ú

¸Ã½»Á÷»ú2U¸ß¶È£¬£¬£¬£¬£¬Ìṩ36¸ö200GµÄÃæ°å¿Ú£¬£¬£¬£¬£¬40¸ö200GµÄFabricÄÚÁª¿Ú£¬£¬£¬£¬£¬4¸öµçÉȺÍ2¸öµçÔ´¡£¡£¡£¡£¡£¡£¡£

NCF£ºRG-X56-96F1½»Á÷»ú

¸Ã½»Á÷»ú4U¸ß¶È£¬£¬£¬£¬£¬Ìṩ96¸ö200GµÄFabricÄÚÁª¿Ú£¬£¬£¬£¬£¬8¸öµçÉȺÍ4¸öµçÔ´¡£¡£¡£¡£¡£¡£¡£

δÀ´stake¹ÙÍøÍøÂ绹»á¼ÌÐøÑз¢¡¢ÍƳö400G¶Ë¿ÚÐÎ̬²úÆ·£¬£¬£¬£¬£¬¾´ÇëÆÚ´ý¡£¡£¡£¡£¡£¡£¡£

½áÓï

stake¹ÙÍøÍøÂ磨֤ȯ´úÂ룺301165£©×÷ΪÐÐÒµÏòµ¼Õߣ¬£¬£¬£¬£¬Ò»Ö±ÖÂÁ¦ÓÚÌṩ¸ßÆ·ÖÊ¡¢¸ß¿É¿¿ÐÔµÄÍøÂç×°±¸Ï¢Õù¾ö¼Æ»®£¬£¬£¬£¬£¬ÒÔÖª×ã¿Í»§¹ØÓÚÖÇËãÖÐÐÄÒ»Ö±Ìá¸ßµÄÐèÇ󡣡£¡£¡£¡£¡£¡£ÔÚÍƳö“ÖÇËÙ“DDC½â¾ö¼Æ»®µÄͬʱ£¬£¬£¬£¬£¬stake¹ÙÍøÍøÂçÒ²ÔÚÆð¾¢Ì½Ë÷ºÍ¿ª·¢¹Å°å×éÍøÖеĶËÍøÓÅ»¯¼Æ»®£¬£¬£¬£¬£¬Í¨¹ý³ä·ÖʹÓ÷þÎñÆ÷ÖÇÄÜÍø¿¨´îÅäÍøÂç×°±¸Ð­ÒéµÄÓÅ»¯£¬£¬£¬£¬£¬ÊµÏÖÕûÍø´ø¿íʹÓÃÂÊÌáÉý£¬£¬£¬£¬£¬×ÊÖú¿Í»§¸ü¿ìÓ­À´AIGCÖÇËãʱ´ú¡£¡£¡£¡£¡£¡£¡£

²Î¿¼ÎÄÏ×£º

[1]Deepak Narayanan, Mohammad Shoeybi, Jared Casper£¬£¬£¬£¬£¬Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM£¬£¬£¬£¬£¬arXiv:2104.04473v5 [cs.CL] 23 Aug 2021