Deep Learning Enabled Semantic Communications - Zhijin Qin

. .

Résumé :

Foreword ix

Preface xi

Acknowledgments xv

Acronyms xvii

Notation xxi

1 Introduction 1

1.1 Conventional Communications versus Semantic Communications 2

1.1.1 Three-level Communications 2

1.1.2 History of Semantic Communications 3

1.2 Introducing Deep Learning to Semantic Communications 4

1.2.1 Deep Learning Basics 4

1.2.2 Deep Learning Enabled Semantic Communications 11

1.3 Semantic Communications for Further Networks 13

References 15

2 Semantic Information Theory 19

2.1 Semantic Entropy 19

2.1.1 Logical Probability Based 19

2.1.2 Synonymous Mapping Based 21

2.1.3 Fuzzy Theory Based 23

2.1.4 Task Based 24

2.2 Semantic Channel Capacity 24

2.2.1 Logical Probability Based 24

2.2.2 Synonymous Mapping Based 25

2.3 Semantic Source Coding Theorem 26

2.3.1 Logical Probability Based 26

2.3.2 Synonymous Mapping Based 27

2.4 Semantic Channel Coding Theorem 28

2.4.1 Logical Probability Based 28

2.4.2 Synonymous Mapping Based 28

2.5 Information Bottleneck 29

2.5.1 Classical Information Bottleneck 29

2.5.2 Knowledge Collision-based Information Bottleneck 30

References 30

3 Joint Semantic-channel Coding for Source Reconstruction 33

3.1 Semantic Communications for Text 34

3.1.1 Joint Semantic-channel Coding for Text 35

3.2 Semantic Communications for Speech 38

3.2.1 Joint Semantic-channel Coding for Speech 39

3.3 Semantic Communications for Image 42

3.3.1 Joint Semantic-channel Coding for Image 42

3.4 Performance Metrics 48

3.4.1 Performance Metrics for Text Accuracy 48

3.4.2 Performance Metrics for Speech Quality 49

3.4.3 Performance Metrics for Image Quality 49

References 52

4 Task-oriented Semantic Communications 55

4.1 Single-modal Task-oriented Semantic Communications 55

4.1.1 Semantic Communications for Machine Translation 56

4.1.2 Semantic Communications for Speech Recognition and Synthesis 59

4.2 Multimodal Task-oriented Semantic Communications 69

4.2.1 Semantic Communication Systems for Visual Question Answering 69

References 74

5 Joint Sensing and Semantic Communications 77

5.1 Introduction and Framework of Joint Sampling and Coding 77

5.1.1 Semantic Sampling 78

5.1.2 Semantic Reconstruction 79

5.2 Joint Semantic Sampling and Coding for Image 79

5.2.1 Semantic-aware Image Compressed Sensing 80

5.2.2 Adaptive Sampling and Semantic-channel Coding 84

5.3 Joint Semantic Sampling and Coding for Video 89

5.3.1 Semantic-based Video Sampling and Reconstruction 92

References 95

6 Semantic Impairments in Communications 97

6.1 JSCC Framework with Semantic Impairments 98

6.2 Source Semantic Impairments Suppression 100

6.2.1 Robust Semantic Communications for Text 100

6.2.2 Robust Semantic Communications for Speech 106

6.2.3 Robust Semantic Communications for Image 113

6.3 Knowledge Base Semantic Impairments Suppression 120

6.3.1 Robust SKB 120

References 125

7 Generative AI-enabled Semantic Communications 129

7.1 Introducing Generative Models to Semantic Communications 129

7.2 Framework of Generative Semantic Communications 131

7.2.1 Main Components of Generative Semantic Communication System 133

7.2.2 Key Interactions and Processes 135

7.3 Demonstration of Generative Semantic Communication for Video Conferencing 136

7.4 Applications of Semantic Communications in Other ...

Sommaire:

Foreword ix

Preface xi

Acknowledgments xv

Acronyms xvii

Notations xxi

1 Introduction?

1.1 Conventional Communications vs. Semantic Communications??

1.1.1 Three-Level Communications?

1.1.2 History of Semantic Communications?

1.2 Introducing Deep Learning to Semantic Communications?

1.2.1 Deep Learning Basics?

1.2.2 Deep Learning Enabled Semantic Communications?

1.3 Semantic Communications for Further Networks?

2 Semantic Information Theory?

2.1 Semantic Entropy?

2.1.1 Logical Probability-based?

2.1.2 Synonymous mapping-based?

2.1.3 Fuzzy Theory-based?

2.1.4 Task-based?

2.2 Semantic Channel Capacity?

2.2.1 Logical Probability-based

2.2.2 Sympuous Mapping based?

2.3 Semantic Source Coding Theorem?

2.3.1 Logical Probability-based?

2.3.2 Sympuous Mapping-based?

2.4 Semantic Channel Coding Theorem?

2.4.1 Logical Probability-based?

2.4.2 Synonymous Mapping-based?

2.5 Information Bottleneck?

2.5.1 Classical Information Bottleneck

2.5.2 Knowledge Collision based Information Bottleneck?

3 Joint Semantic-Channel Coding for Source Reconstruction?

3.1 Semantic Communications for Text?

3.1.1 Joint Semantic-Channel Coding for Text?

3.2 Semantic Communications for Speech?

3.2.1 Joint Semantic-Channel Coding for Speech?

3.3 Semantic Communications for Image?

3.3.1 Joint Semantic-Channel Coding for Image?

3.4 Performance Metrics?

3.4.1 Performance Metrics for Text Accuracy?

3.4.2 Performance Metrics for Speech Quality?

3.4.3 Performance Metrics for Image Quality?

4 Task-Oriented Semantic Communications?

4.1 Single-Modal Task-Oriented Semantic Communications?

4.1.1 Semantic Communications for Machine Translation?

4.1.2 Semantic Communications for Speech Recognition and Synthesis?

4.2 Multimodal Task-Oriented Semantic Communications?

4.2.1 Semantic Communication Systems for Visual Question Answering?

5 Joint Sensing and Semantic Communications?

5.1 Introduction and Framework of Joint Sampling and Coding?

5.2 Joint Semantic Sampling and Coding for Image?

5.2.1 Semantic-aware Image Compressed Sensing?

5.2.2 Adaptive Sampling and Semantic-Channel Coding?

5.3 Joint Semantic Sampling and Coding for Video?

5.3.1 Semantic-based Video Sampling and Reconstruction?

6 Semantic Impairments in Communications?

6.1 JSCC Framework with Semantic Impairments?

6.2 Source Semantic Impairments Suppression?

6.2.1 Robust Semantic Communications for Text?

6.2.2 Robust Semantic Communications for Speech?

6.2.3 Robust Semantic Communications for Image?

6.3 Knowledge Base Semantic Impairments Suppression?

6.3.1 Robust Semantic Knowledge Base?

7 Generative AI Enabled Semantic Communications?

7.1 Introducing Generative Models to Semantic Communications?

7.2 Framework of Generative Semantic Communications?

7.2.1 Main Components of Generative Semantic Communication System?

7.2.2 Key Interactions and Processes?

7.3 Demonstration of Generative Semantic Communication for Video Conferencing?

7.4 Applications of Semantic Communications in Other Scenarios?

7.4.1 Immersive Communications?

7.4.2 Autonomous Driving?

7.4.3 Smart Cities?

7.4.4 Satellite Networks?

8 Conclusions and Challenges?