webrtc 视频的采集,编码,发送流程详细分析(希望对底层深入了解的朋友有所帮助)

本文深入剖析WebRTC视频采集、编码及发送流程,从Java层的PeerConnectionClient到Native层的VideoTrack创建,再到TurnPort的构建过程,以及视频编码器的初始化与发送线程的启动。涵盖AndroidVideoTrackSource的创建、VideoTrack与源的关联、TurnPort的产生机制等关键技术细节。

----------------------------------------------------------------------------------------------------------------------------------------

一分钟快速搭建 rtmpd 服务器: https://blog.csdn.net/freeabc/article/details/102880984

软件下载地址: http://www.qiyicc.com/download/rtmpd.rar

github 地址:https://github.com/superconvert/smart_rtmpd

-----------------------------------------------------------------------------------------------------------------------------------------

webrtc 视频的采集,编码,发送流程详细分析

 

我写文章一般是两个思路:
1. 下一步要调用什么对象的方法
2.  这一步的对象,怎么关联到下一步的对象的流程分析
这一步的流程主要阐述怎么关联下一步的对象的流程分析,当然这一步做了什么具体的工作,不能
详细展示,否则,太庞大了,需要各位朋友针对重点的部分,自己揣摩了。

//-----------------------------------------------------------------------------
//
// 视频编码器的创建以及与源的关联
//
//-----------------------------------------------------------------------------
1. Java
PeerConnectionClient::createPeerConnectionInternal
    peerConnection.addTrack(createVideoTrack(videoCapturer), mediaStreamLabels);

    1.1. createVideoTrack 的建立
    PeerConnectionClient::createVideoTrack(VideoCapturer capturer)
        //---------------------------------------------------------
        // 这个里面会在 Jni 层创建一个 AndroidVideoTrackSource 对象
        //---------------------------------------------------------
        videoSource = factory.createVideoSource(capturer.isScreencast());
        //--------------------------------------------------------
        // 这个会为 source 绑定一个 track 对象
        //---------------------------------------------------------
        localVideoTrack = factory.createVideoTrack(VIDEO_TRACK_ID, videoSource);

    1.2
    PeerConnectionFactory::createVideoTrack(String id, VideoSource source)
        checkPeerConnectionFactoryExists();
        return new VideoTrack(nativeCreateVideoTrack(nativeFactory, id, source.getNativeVideoTrackSource()));

    1.3
    nativeCreateVideoTrack

    1.4
    Java_org_webrtc_PeerConnectionFactory_nativeCreateVideoTrack(JNIEnv* env, jclass jcaller, jlong factory, jstring id, jlong nativeVideoSource) {
        return JNI_PeerConnectionFactory_CreateVideoTrack(env, factory, base::android::JavaParamRef<jstring>(env, id), nativeVideoSource);
    }

    1.5 ./sdk/android/src/jni/pc/peer_connection_factory.cc
    JNI_PeerConnectionFactory_CreateVideoTrack(JNIEnv* jni, jlong native_factory, const JavaParamRef<jstring>& id, jlong native_source) {
        rtc::scoped_refptr<VideoTrackInterface> track = PeerConnectionFactoryFromJava(native_factory)->CreateVideoTrack(
        JavaToStdString(jni, id), reinterpret_cast<VideoTrackSourceInterface*>(native_source));
        return jlongFromPointer(track.release());
    }
    native_source 就是 AndroidVideoTrackSource

    1.6 ./pc/peer_connection_factory.cc
    rtc::scoped_refptr<VideoTrackInterface> PeerConnectionFactory::CreateVideoTrack(const std::string& id, VideoTrackSourceInterface* source) {
        RTC_DCHECK(signaling_thread_->IsCurrent());
        rtc::scoped_refptr<VideoTrackInterface> track(VideoTrack::Create(id, source, worker_thread_));
        return VideoTrackProxy::Create(signaling_thread_, worker_thread_, track);
    }

    1.7 ./pc/video_track.cc
    rtc::scoped_refptr<VideoTrack> VideoTrack::Create(const std::string& id, VideoTrackSourceInterface* source, rtc::Thread* worker_thread) {
        rtc::RefCountedObject<VideoTrack>* track = new rtc::RefCountedObject<VideoTrack>(id, source, worker_thread);
        return track;
    }

从上面流程看出 VideoTrack 包含了 AndroidVideoTrackSource

 

2. Java
PeerConnection::addTrack
    RtpSender newSender = nativeAddTrack(track.getNativeMediaStreamTrack(), streamIds);

3. 
Java_org_webrtc_PeerConnection_nativeAddTrack
    JNI_PeerConnection_AddTrack

4. ./sdk/android/src/jni/pc/peer_connection.cc

JNI_PeerConnection_AddTrack
    ExtractNativePC(jni, j_pc)->AddTrack(reinterpret_cast<MediaStreamTrackInterface*>(native_track),
    JavaListToNativeVector<std::string, jstring>(jni, j_stream_labels, &JavaToNativeString));

5.

PeerConnection::AddTrack(rtc::scoped_refptr<MediaStreamTrackInterface> track, const std::vector<std::string>& stream_ids)
    auto sender_or_error = (IsUnifiedPlan() ? AddTrackUnifiedPlan(track, stream_ids) : AddTrackPlanB(track, stream_ids));

6.

PeerConnection::AddTrackPlanB(rtc::scoped_refptr<MediaStreamTrackInterface> track, const std::vector<std::string>& stream_ids)
    auto new_sender = CreateSender(media_type, track->id(), track, adjusted_stream_ids, {});
    if (track->kind() == MediaStreamTrackInterface::kAudioKind) {
      new_sender->internal()->SetMediaChannel(voice_media_channel());
      GetAudioTransceiver()->internal()->AddSender(new_sender);
      const RtpSenderInfo* sender_info = FindSenderInfo(local_audio_sender_infos_, new_sender->internal()->stream_ids()[0], track->id());
      if (sender_info) {
        new_sender->internal()->SetSsrc(sender_info->first_ssrc);
      }
    } else {
      RTC_DCHECK_EQ(MediaStreamTrackInterface::kVideoKind, track->kind());
      new_sender->internal()->SetMediaChannel(video_media_channel());
      GetVideoTransceiver()->internal()->AddSender(new_sender);
      const RtpSenderInfo* sender_info = FindSenderInfo(local_video_sender_infos_, new_sender->internal()->stream_ids()[0], track->id());
      if (sender_info) {
        new_sender->internal()->SetSsrc(sender_info->first_ssrc);
      }
    }
    return rtc::scoped_refptr<RtpSenderInterface>(new_sender);

7.

PeerConnection::CreateSender(cricket::MediaType media_type, const std::string& id, rtc::scoped_refptr<MediaStreamTrackInterface> track,
    const std::vector<std::string>& stream_ids, const std::vector<RtpEncodingParameters>& send_encodings)
    sender = RtpSenderProxyWithInternal<RtpSenderInternal>::Create(signaling_thread(), VideoRtpSender::Create(worker_thread(), id, this));
    NoteUsageEvent(UsageEvent::VIDEO_ADDED);
    bool set_track_succeeded = sender->SetTrack(track); --->
    RTC_DCHECK(set_track_succeeded);
    sender->internal()->set_stream_ids(stream_ids);
    sender->internal()->set_init_send_encodings(send_encodings);

8.

VideoRtpSender::SetTrack(MediaStreamTrackInterface* track)
RtpSenderBase::SetTrack(MediaStreamTrackInterface* track)
    SetSend();

9.

VideoRtpSender::SetSend()
    cricket::VideoOptions options;
    VideoTrackSourceInterface* source = video_track()->GetSource();
    if (source) {
        options.is_screencast = source->is_screencast();
        options.video_noise_reduction = source->needs_denoising();
    }
    switch (cached_track_content_hint_) {
    case VideoTrackInterface::ContentHint::kNone:
      break;
    case VideoTrackInterface::ContentHint::kFluid:
      options.is_screencast = false;
      break;
    case VideoTrackInterface::ContentHint::kDetailed:
    case VideoTrackInterface::ContentHint::kText:
      options.is_screencast = true;
      break;
    }
    bool success = worker_thread_->Invoke<bool>(RTC_FROM_HERE, [&] {
    return video_media_channel()->SetVideoSend(ssrc_, &options, video_track());

10.

WebRtcVideoChannel::SetVideoSend(uint32_t ssrc, const VideoOptions* options, rtc::VideoSourceInterface<webrtc::VideoFrame>* source)
    const auto& kv = send_streams_.find(ssrc);
    if (kv == send_streams_.end()) {
        // Allow unknown ssrc only if source is null.
        RTC_CHECK(source == nullptr);
        RTC_LOG(LS_ERROR) << "No sending stream on ssrc " << ssrc;
        return false;
    }
    return kv->second->SetVideoSend(options, source);

    而 send_streams_ 就是 WebRtcVideoChannel::WebRtcVideoSendStream 参见下面的
    《WebRtcVideoChannel 对象 AddSendStream (WebRtcVideoSendStream send_streams_) 的过程》
    WebRtcVideoChannel::AddSendStream(const StreamParams& sp)11.

WebRtcVideoChannel::WebRtcVideoSendStream::SetVideoSend(const VideoOptions* options,
    rtc::VideoSourceInterface<webrtc::VideoFrame>* source)
    ReconfigureEncoder();   ---> 流程 12
    source_ = source;
    if (source && stream_) {
        !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        这段语句是把源与编码器对象关联到一起!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        stream_->SetSource(this, GetDegradationPreference());
    }

    stream_ 就是 VideoSendStream 对象,具体创建过程参见
    void WebRtcVideoChannel::WebRtcVideoSendStream::RecreateWebRtcStream() 中的
    stream_ = call_->CreateVideoSendStream(std::move(config), parameters_.encoder_config.Copy());

   stream_->SetSource 的接口继续分析

11.

    11.1
    void VideoSendStream::SetSource(rtc::VideoSourceInterface<webrtc::VideoFrame>* source,
        const DegradationPreference& degradation_preference) {
        RTC_DCHECK_RUN_ON(&thread_checker_);
        video_stream_encoder_->SetSource(source, degradation_preference);
    }

    11.2
    void VideoStreamEncoder::SetSource(rtc::VideoSourceInterface<VideoFrame>* source,
        const DegradationPreference& degradation_preference)
        // 这个里面做 source 与 encoder 的关联 
        source_proxy_->SetSource(source, degradation_preference);

        而 source_proxy_ 是 VideoStreamEncoder::VideoStreamEncoder 构造时创建的
        ./video/video_stream_encoder.cc
        VideoStreamEncoder::VideoStreamEncoder(
            Clock* clock,
            uint32_t number_of_cores,
            VideoStreamEncoderObserver* encoder_stats_observer,
            const VideoStreamEncoderSettings& settings,
            std::unique_ptr<OveruseFrameDetector> overuse_detector,
            TaskQueueFactory* task_queue_factory) 
            : source_proxy_(new VideoSourceProxy(this)),

        VideoSourceProxy 的 video_stream_encoder_ 就是 VideoStreamEncoder
        ./video/video_stream_encoder.cc
        explicit VideoSourceProxy(VideoStreamEncoder* video_stream_encoder)
            : video_stream_encoder_(video_stream_encoder),
            degradation_preference_(DegradationPreference::DISABLED),
            source_(nullptr),
            max_framerate_(std::numeric_limits<int>::max()) {}


    11.3 这个地方把 VideoStreamEncoder 和 Source 相关联
    void VideoSourceProxy::SetSource(rtc::VideoSourceInterface<VideoFrame>* source, 
        const DegradationPreference& degradation_preference) {
        // Called on libjingle's worker thread.
        RTC_DCHECK_RUN_ON(&main_checker_);
        rtc::VideoSourceInterface<VideoFrame>* old_source = nullptr;
        rtc::VideoSinkWants wants;
        {
          rtc::CritScope lock(&crit_);
          degradation_preference_ = degradation_preference;
          old_source = source_;
          source_ = source;
          wants = GetActiveSinkWantsInternal();
        }

        if (old_source != source && old_source != nullptr) {
          old_source->RemoveSink(video_stream_encoder_);
        }

        if (!source) {
          return;
        }
        // source 就是上面设置过来的 WebRtcVideoChannel::WebRtcVideoSendStream 对象
        source->AddOrUpdateSink(video_stream_encoder_, wants);        
     }


     11.4
     void WebRtcVideoChannel::WebRtcVideoSendStream::AddOrUpdateSink(
         rtc::VideoSinkInterface<webrtc::VideoFrame>* sink, const rtc::VideoSinkWants& wants)
         encoder_sink_ = sink;
         source_->AddOrUpdateSink(encoder_sink_, wants);


     11.5
     void VideoTrack::AddOrUpdateSink(rtc::VideoSinkInterface<VideoFrame>* sink, const rtc::VideoSinkWants& wants) {
         RTC_DCHECK(worker_thread_->IsCurrent());
         VideoSourceBase::AddOrUpdateSink(sink, wants);
         rtc::VideoSinkWants modified_wants = wants;
         modified_wants.black_frames = !enabled();
         // video_source_ 就是 AndroidVideoTrackSource(AdaptedVideoTrackSource) 对象
         video_source_->AddOrUpdateSink(sink, modified_wants);
      }

     video_source_ 就是 AndroidVideoTrackSource 对象 
     (void AndroidVideoTrackSource::AddOrUpdateSink)

     11.6
     void AdaptedVideoTrackSource::AddOrUpdateSink(rtc::VideoSinkInterface<webrtc::VideoFrame>* sink, const rtc::VideoSinkWants& wants) {

          !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
          就这个地方,把视频源对象 AndroidVideoTrackSource 与 视频编码对象 VideoStreamEncoder 关联一起的
          broadcaster_.AddOrUpdateSink(sink, wants);
          !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

          OnSinkWantsChanged(broadcaster_.wants());
     }

12.

WebRtcVideoChannel::WebRtcVideoSendStream::ReconfigureEncoder()
    webrtc::VideoEncoderConfig encoder_config = CreateVideoEncoderConfig(codec_settings.codec);
    encoder_config.encoder_specific_settings = ConfigureVideoEncoderSettings(codec_settings.codec);
    stream_->ReconfigureVideoEncoder(encoder_config.Copy());


    //-----------------------------------------------------------
    // 而 stream_ 就是一个 VideoSendStream 对象,是由下面流程产生的
    //-----------------------------------------------------------

    1. WebRtcVideoChannel::WebRtcVideoSendStream::RecreateWebRtcStream()
        stream_ = call_->CreateVideoSendStream(std::move(config), parameters_.encoder_config.Copy());

    2. webrtc::VideoSendStream* Call::CreateVideoSendStream(webrtc::VideoSendStream::Config config, VideoEncoderConfig encoder_config)
        std::unique_ptr<FecController> fec_controller = config_.fec_controller_factory ? 
            config_.fec_controller_factory->CreateFecController() : std::make_unique<FecControllerDefault>(clock_);
        return CreateVideoSendStream(std::move(config), std::move(encoder_config), std::move(fec_controller));

     3. webrtc::VideoSendStream* Call::CreateVideoSendStream(webrtc::VideoSendStream::Config config, 
        VideoEncoderConfig encoder_config, std::unique_ptr<FecController> fec_controller)
        VideoSendStream* send_stream = new VideoSendStream(clock_, num_cpu_cores_, module_process_thread_.get(), 
        task_queue_factory_, call_stats_.get(), transport_send_ptr_, bitrate_allocator_.get(),
            video_send_delay_stats_.get(), event_log_, std::move(config), std::move(encoder_config), 
        suspended_video_send_ssrcs_, suspended_video_payload_states_, std::move(fec_controller));

13.

VideoSendStream::ReconfigureVideoEncoder
    video_stream_encoder_->ConfigureEncoder(std::move(config), config_.rtp.max_packet_size - CalculateMaxHeaderSize(config_.rtp)); --->

    //----------------------------------------------------------
    // video_stream_encoder_ 就是 VideoStreamEncoder 对象,产生流程如下
    //----------------------------------------------------------
    VideoSendStream::VideoSendStream(Clock* clock, int num_cpu_cores, ProcessThread* module_process_thread, TaskQueueFactory* task_queue_factory,
        CallStats* call_stats, RtpTransportControllerSendInterface* transport, BitrateAllocatorInterface* bitrate_allocator, SendDelayStats* send_delay_stats,
        RtcEventLog* event_log, VideoSendStream::Config config, VideoEncoderConfig encoder_config, const std::map<uint32_t, RtpState>& suspended_ssrcs,
        const std::map<uint32_t, RtpPayloadState>& suspended_payload_states, std::unique_ptr<FecController> fec_controller) 
        video_stream_encoder_ = CreateVideoStreamEncoder(clock, task_queue_factory, num_cpu_cores, &stats_proxy_, config_.encoder_settings);

14.

VideoStreamEncoder::ConfigureEncoder(VideoEncoderConfig config, size_t max_data_payload_length)
    ReconfigureEncoder(); --->

15.

VideoStreamEncoder::ReconfigureEncoder()
    std::vector<VideoStream> streams =
      encoder_config_.video_stream_factory->CreateEncoderStreams(last_frame_info_->width, last_frame_info_->height, encoder_config_); 

    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    这个地方就是创建了编码器对象
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    encoder_ = settings_.encoder_factory->CreateVideoEncoder(encoder_config_.video_format); --->

    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    这个函数还有对 encoder_ 的初始化工作 参见下面的视频编码器的初始化
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    if (encoder_->InitEncode(&send_codec_, VideoEncoder::Settings(settings_.capabilities, number_of_cores_, max_data_payload_length)) != 0)

16.

VideoEncoderFactoryWrapper::CreateVideoEncoder(const SdpVideoFormat& format)
    ScopedJavaLocalRef<jobject> encoder = Java_VideoEncoderFactory_createEncoder(jni, encoder_factory_, j_codec_info); --->
    return JavaToNativeVideoEncoder(jni, encoder);

17.

    17.1
    static base::android::ScopedJavaLocalRef<jobject> Java_VideoEncoderFactory_createEncoder(JNIEnv*
        env, const base::android::JavaRef<jobject>& obj, const base::android::JavaRef<jobject>& info) {
        jclass clazz = org_webrtc_VideoEncoderFactory_clazz(env);
        CHECK_CLAZZ(env, obj.obj(), org_webrtc_VideoEncoderFactory_clazz(env), NULL);
        jni_generator::JniJavaCallContextChecked call_context;
        call_context.Init<base::android::MethodID::TYPE_INSTANCE>(env, clazz,
            "createEncoder",
            "(Lorg/webrtc/VideoCodecInfo;)Lorg/webrtc/VideoEncoder;",
            &g_org_webrtc_VideoEncoderFactory_createEncoder);
        jobject ret = env->CallObjectMethod(obj.obj(), call_context.base.method_id, info.obj());
        return base::android::ScopedJavaLocalRef<jobject>(env, ret);
    }

    17.2
    JavaToNativeVideoEncoder(JNIEnv* jni, const JavaRef<jobject>& j_encoder) {
        const jlong native_encoder = Java_VideoEncoder_createNativeVideoEncoder(jni, j_encoder);
        VideoEncoder* encoder;
        if (native_encoder == 0) {
            encoder = new VideoEncoderWrapper(jni, j_encoder);
        } else {
            encoder = reinterpret_cast<VideoEncoder*>(native_encoder);
       }
       return std::unique_ptr<VideoEncoder>(encoder);

 VideoEncoderWrapper 被创建   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!      

//-----------------------------------------------------------------------------
//
// 视频编码器的初始化
//
//-----------------------------------------------------------------------------
1. 
这个的流程参见上面的  流程 15

VideoStreamEncoder::ReconfigureEncoder()
    if (encoder_->InitEncode(&send_codec_, VideoEncoder::Settings(settings_.capabilities, number_of_cores_, max_data_payload_length)) != 0)

2.

VideoEncoderWrapper::InitEncode(const VideoCodec* codec_settings, const Settings& settings)
    JNIEnv* jni = AttachCurrentThreadIfNeeded();
    codec_settings_ = *codec_settings;
    capabilities_ = settings.capabilities;
    number_of_cores_ = settings.number_of_cores;
    num_resets_ = 0;
    {
        rtc::CritScope lock(&encoder_queue_crit_);
        encoder_queue_ = TaskQueueBase::Current();
    }
    return InitEncodeInternal(jni);

3.

VideoEncoderWrapper::InitEncodeInternal(JNIEnv* jni)
    ScopedJavaLocalRef<jobject> capabilities = Java_Capabilities_Constructor(jni, capabilities_->loss_notification);
    ScopedJavaLocalRef<jobject> settings = Java_Settings_Constructor(
        jni, number_of_cores_, codec_settings_.width, codec_settings_.height,
        static_cast<int>(codec_settings_.startBitrate),
        static_cast<int>(codec_settings_.maxFramerate),
        static_cast<int>(codec_settings_.numberOfSimulcastStreams),
        automatic_resize_on, capabilities);
    ScopedJavaLocalRef<jobject> callback = Java_VideoEncoderWrapper_createEncoderCallback(jni, jlongFromPointer(this));
    int32_t status = JavaToNativeVideoCodecStatus(jni, Java_VideoEncoder_initEncode(jni, encoder_, settings, callback));
    RTC_LOG(LS_INFO) << "initEncode: " << status;
    encoder_info_.supports_native_handle = true;
    encoder_info_.implementation_name = GetImplementationName(jni);
    encoder_info_.scaling_settings = GetScalingSettingsInternal(jni);
    encoder_info_.is_hardware_accelerated = IsHardwareVideoEncoder(jni, encoder_);
    encoder_info_.has_internal_source = false;
    if (status == WEBRTC_VIDEO_CODEC_OK) {
        initialized_ = true;
    }
    return status;

4.

Java_VideoEncoder_initEncode(JNIEnv* env, const base::android::JavaRef<jobject>& obj, const base::android::JavaRef<jobject>& settings,
    const base::android::JavaRef<jobject>& encodeCallback)
    jclass clazz = org_webrtc_VideoEncoder_clazz(env);
    CHECK_CLAZZ(env, obj.obj(), org_webrtc_VideoEncoder_clazz(env), NULL);
    jni_generator::JniJavaCallContextChecked call_context;
    call_context.Init<base::android::MethodID::TYPE_INSTANCE>(env, clazz, "initEncode",
        "(Lorg/webrtc/VideoEncoder$Settings;Lorg/webrtc/VideoEncoder$Callback;)Lorg/webrtc/VideoCodecStatus;",
        &g_org_webrtc_VideoEncoder_initEncode);
    jobject ret = env->CallObjectMethod(obj.obj(), call_context.base.method_id, settings.obj(), encodeCallback.obj());

5. Java

HardwareVideoEncoder::initEncode(Settings settings, Callback callback)
    initEncodeInternal

6. Java

HardwareVideoEncoder::initEncodeInternal
    // 创建编码器,并设置参数
    codec = mediaCodecWrapperFactory.createByCodecName(codecName);
    // 开启编码数据发送线程
    outputThread = createOutputThread();
    outputThread.start();

//-----------------------------------------------------------------------------------------
// 编码流程分析
//-----------------------------------------------------------------------------------------
1. Java

ScreenCapturerAndroid::onFrame(VideoFrame frame)
    capturerObserver.onFrameCaptured(frame);

2. Java

VideoSource::CapturerObserver::onFrameCaptured(VideoFrame frame)
    final VideoProcessor.FrameAdaptationParameters parameters = nativeAndroidVideoTrackSource.adaptFrame(frame);
    VideoFrame adaptedFrame = VideoProcessor.applyFrameAdaptationParameters(frame, parameters);
    if (adaptedFrame != null) {
        nativeAndroidVideoTrackSource.onFrameCaptured(adaptedFrame);
        adaptedFrame.release();
    }

    这里的 nativeAndroidVideoTrackSource 就是 AndroidVideoTrackSource

2.1
    nativeCreateVideoSource(nativeFactory, isScreencast, alignTimestamps)

2.2
    Java_org_webrtc_PeerConnectionFactory_nativeCreateVideoSource

2.3 ./sdk/android/src/jni/pc/peer_connection_factory.cc
    JNI_PeerConnectionFactory_CreateVideoSource    
        return jlongFromPointer(CreateVideoSource(jni, factory->signaling_thread(), factory->worker_thread(), is_screencast, align_timestamps));

2.4 ./jni/pc/video.cc
    void* CreateVideoSource(JNIEnv* env, rtc::Thread* signaling_thread, rtc::Thread* worker_thread, jboolean is_screencast, jboolean align_timestamps) 
        rtc::scoped_refptr<AndroidVideoTrackSource> source(new rtc::RefCountedObject<AndroidVideoTrackSource>(signaling_thread, env, is_screencast,
            align_timestamps));

3.

NativeAndroidVideoTrackSource::onFrameCaptured(VideoFrame frame)
    nativeOnFrameCaptured(nativeAndroidVideoTrackSource, frame.getRotation(), frame.getTimestampNs(), frame.getBuffer());

4.

Java_org_webrtc_NativeAndroidVideoTrackSource_nativeOnFrameCaptured
    AndroidVideoTrackSource* native = reinterpret_cast<AndroidVideoTrackSource*>(nativeAndroidVideoTrackSource);
    CHECK_NATIVE_PTR(env, jca
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值